I'd like to use some of the more advanced features of the REST Dataset in Azure Data Factory (mainly pagination), but I can't see where to use Certificate Authentication. I have an HTTP Dataset working well with Certificate Authentication, but the only options seem to be:
Anonymous
Basic
AAD Service Principal
Managed Identity
Is there some way to use the AAD Service Principal or Managed Identity with a certificate? Maybe I can do this using the Advanced settings?
The docs say that the HTTP connector is "less functional comparing to REST connector" but this seems to be a pretty big omission from the REST version.
Is there some way to use the AAD Service Principal or Managed Identity
with a certificate? Maybe I can do this using the Advanced settings?
SP and MI can't be applied for the certificate authentication, for AAD instead. please refer to this simple blog.
Based on the REST Connector document,it only supports above 4 types of authenticate options. If you want to carry certificate with request,please use HTTP connector.
Related
I am working on creating a flow where I get JSON data from a rest api authenticating with the managed identity of my ADF instance and copy the data to a Kusto cluster. To do this, I am following the instructions here: https://learn.microsoft.com/en-us/azure/data-factory/connector-http?tabs=data-factory
However, when I go to create a linked service I do not see any way to authenticate against the API using my managed identity: http linked service auth options
I was expecting something like the options given in the Web task in ADF, where I am allowed to selected managed identities:web task auth options
HTTP connector supports only Anonymous, Basic, Digest, Windows, and ClientCertificate as authentication types. To use Managed Identity authentication type, you can use Rest linked service.
Search for REST in the available list of linked service and select REST connector.
You can select Managed Identity as auth type.
Reference: MS doc on Data Fcatory - REST connector.
Many of my (confidential) apps are talking to each other via the client credential flow.
They request a token from the Azure Identity platform and use this token to authenticate against another app.
A while ago I used client secrets to do so, but later I read that this is not recommended for production environments.
For this reason I changed to self-signed certificates that are valid a longer time.
Those certificates are generated by myself with Azure Keyvault.
However, also this is not recommended.
Microsoft states that that in production environments you should use certificates that are signed by an official CA.
If I now use Lets encrypt, this will expire all three months what is also not such a nice solution.
My questions:
Why is the client secret not recommended in production environments?
Why is the self-signed certificate a problem? I do understand this in matters of HTTPS, but where is the security breach if its used for client credential flow? In my case I am the owner of the app and the app registration.
Do I need to buy a certificate that is one-year valid to do it "the right way"?
Do you have any source of best practices here?
• Client secrets include application credentials, SSH keys, API keys, database passwords, encryption keys, connection strings and so on to connect various resources and access the data or functionality for achieving the designated purpose of that application. Thus, if these are breached, they can put your application at great risk of compromise. Also, the client secret generated in Azure AD and used in APIs for connecting to Azure AD for authentication and authorization purpose is listed and mentioned in unencrypted form in the API code itself. Though, we have an option to store that secret in a key vault and refer to that secret through either managed identity or RBAC assignments, but their credentials too can fall in wrong hands and let the application be vulnerable if the managed identity is a user assigned or even if then access scope of the secret is not well defined according to the required specific need. Thus, client secret is not recommended to be used in a production API.
• In client credentials flow, applications are directly granted permissions by an administrator to perform a certain action regarding the API to be called through it via certificate or federated credentials. Thus, when using a self-signed certificate in client credentials grant scenario, the administrator has granted the daemon app requesting access to other API all the required privileges regarding accessibility of code, API, permissions, data, etc. which can result in poor validation and misuse as the it is very easy to generate a certificate’s key pair without reasonable entropy. Also, protecting the private key of the key pair appropriately to its use and strong validation of the same is not promised in a self-signed certificate due to which it is not recommended in client credentials flow.
• For best practices regarding web app service deployment, please refer to the documentation link below: -
https://learn.microsoft.com/en-us/azure/app-service/security-recommendations#general
It explains the best security recommendations for deploying a web app service.
I'm wondering what is the correct way to grant service accounts access to AzureDevOps. Ex: access to source code, API's, etc.
Since "Basic Authentication" was deactivated, I've been using personal access tokens but it seems awkward to login using a service account, generating the PAT, and so on. Also, since they are limited in time, we have to ensure renewal schedules/reminders for each connexions.
I am on the wrong track? Is there a more "natural" way to do it? Is SSH more adapted for this?
I've found this guide which helps but I can't find how (for example) granting a third-party application access to a project's repository.
https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/authentication-guidance?view=azure-devops
PS: Service Connections and Service Hooks are not what I'm looking for.
What you are looking for probably is OAuth 2.0 Client Credentials Flow for Azure DevOps. Unfortunately Client Credentials Flow is not yet supported on Azure DevOps. See this thread Client Credentials Flow for Azure DevOps.
But, You can check out OAuth 2.0 auth code flow,which is now supported on Azure Devops. Please check out this detailed tutorial Authorize access to REST APIs with OAuth 2.0. This is an example to implement OAuth 2.0 auth code flow.
There are also Codes samples provided in above web link you mentioned in the question. You can checkout these samples to learn their authentication mechanism, and choose one that suits you most.
I tried Generic type service connection, which seems to support only basic authentication (username/password).
Appreciate any guidance!!
Service Connection Screenshot
We don't currently support MFA with the connector.
You could take a look at official doc here-- Manage Conditional Access to Azure DevOps
Azure DevOps enforces the policy for usage of personal access tokens
(PATs), alternate authentication, OAuth, and SSH keys.
For third-party client flow, like using a PAT with git.exe, we only
support IP fencing policies - we don't support MFA policies.
In cloud foundry how can I get the client_id and client_secret code.
This will be used in Basic authentication instead of passing the user and password for respective api calls
In general, this is something that you would get from your platform administrator. He or she would be able to provide you with a properly configured client id and client secret to fit your needs. If you are not the administrator, then you won't be able to do this.
If you are an administrator, read on. There are many existing client and secret pairs within a Cloud Foundry platform. It's also possible, and suggested, to create custom client and secret pairs for use with non-platform apps. You shouldn't use a platform client for your custom app, you should use your own custom client, that way if the client is compromised you can delete it or change the secret.
All of this, viewing & managing client data, is done through UAA, so I'd recommend starting with the docs on UAA. Make sure that you understand the concepts. You may even want to take a step further back and review concepts of OAuth2 as well. Understanding OAuth2 will make working with UAA much easier.
https://docs.cloudfoundry.org/uaa/uaa-overview.html
After that, you'll need the uaac (i.e. UAA client) installed.
https://github.com/cloudfoundry/cf-uaac
Once you're familiar with UAA & have the client installed, this doc on how to manage clients should provide you with instructions to view or create a new client.
https://github.com/cloudfoundry/uaa/blob/master/docs/Sysadmin-Guide.rst#manage-client-registrations
You will need admin role to be able to create a client or get its details.
Refer https://docs.cloudfoundry.org/api/uaa/version/4.7.1/index.html#clients to see various api to manage and create clients.
You can also use the uaa client to get the client details.
Refer https://docs.cloudfoundry.org/uaa/uaa-user-management.html for more info.