I've assigned the SQL Server Integration Services integration runtimes on Azure Data Factory should be joined to a virtual network Azure Policy to my resource group where I'm deploying a Data Factory instance.
I've VNet injected the SSIS IR using the express method This worked + confirmed that all the requirements are fulfilled, but the policy evaluation returns a state of not complianet:
Policy condition criteria
ARM template exported for my ADF setup.
Any idea what I’m doing wrong? I suspect this might be an issue with the Policy definition itself (using the built-in policy).
Related
We have a release pipeline that is failing with following message:
resource ID for resource type 'Microsoft.Web/Sites' and resource name
'appservicename'. Error: Could not fetch access token for Managed
Service Principal. Please configure Managed Service Identity (MSI) for
virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400,
status message: Bad Request
We have 2 different service connections:
Azure Resource Manager using service principal authentication
Azure Resource Manager using managed identity authentication
The first one works like a charm. However, because the developer wanted to limit admin access on the Azure AD, he tried creating a managed identity authentication service connection which at first glance, since it allowed us to select the App Service, appeared to indicate it's working, until an actual deployment was triggered and it failed per the error message above.
After numerous searches online, I think this answer may be the clue to why this is failing with the managed identity authentication service connection yet succeeding with the service principal connection just fine.
I just want to confirm, is this truly the case? that a hosted agent doesn't support MSI based authentication, which is what we are using… or has that changed?
We are indeed using Microsoft agent pool.
It doesn't make sense for our app service to use a VM at this time. The use case just isn't applicable for the dashboards we have.
As it is written in the docs:
You are required to use a self-hosted agent on an Azure VM in order to use managed service identity
I assume that it was alway like that. Here we are talking abut MSI assigned to VM which serves as build agent. Not MSI which is identity of App Service. Why? Service Connection is an abstraction which makes easy authentication to your Azure Subscription. So it gives identity to VM and then when your perform some action against your Azure thanks to MSI Azure know that can perform that action. Another aption is authentication via Service Principal, but thi can be done from any VM (inlcuding MS Hosted) because it relies on Client Id and Client secret which is kept in service connections. And MSI have to be assigned to particular VM which cannot be done with MS Hosted agents.
We need to ensure that the VM hosting the Self Hosted Integration Runtime (SHIR) can be supported without risking exposure of the data being processed by the SHIR - which may be sensitive in nature.
What information related to data sources is cached locally on the VM hosting a Self Hosted Integration Runtime (SHIR)? For example, if SHIR is pulling data from an on-prem SQL Server, is it possible that data queried could be exposed in error / event logs on the VM?
Thanks.
Matt
According to the documentation, this is not a risk
Azure Data Factory including Azure Integration Runtime and Self-hosted Integration Runtime does not store any temporary data, cache data or logs except for linked service credentials for cloud data stores, which are encrypted by using certificates.
Reference: https://learn.microsoft.com/en-us/azure/data-factory/data-movement-security-considerations
I have copy activity, where source is Oracle on premises connected through Self-hosted IR and destination is Microsoft Synapse connected via Azure Run-time. These run-time is defined in connections (Linked Services).
But while execution pipeline is using Self Hosted Run-time through-out and overriding the run-time of Azure Synapse. And because of that connection is failing.
Is this default behavior? Can't I run pipeline with 2 different run-time.
Thanks #wBob but I am sorry that is not true, I found the answer at Microsoft documentation.
Copying between a cloud data source and a data source in private network: if either source or sink linked service points to a self-hosted IR, the copy activity is executed on that self-hosted Integration Runtime.
Ref: https://learn.microsoft.com/en-us/azure/data-factory/concepts-integration-runtime#determining-which-ir-to-use
Integration runtimes are defined at the linked service level. So you should have a linked service definition for your Oracle database and a separate linked service definition for your Azure Synapse Analytics (formerly known as Azure SQL Data Warehouse). So you can specify different integration runtimes, eg here's an example:
NB Azure Synapse Analytics is using the Autoresolve runtime and does not need a self-hosted integration runtime (SHIR) as it is a native PaaS service. Are you getting a specific error? If so, please post details.
I am trying to configure Azure Key Vault and setup Managed Identities for use in CI/CD pipeline for Azure Dev Ops.
I have looked around in MSDN documentation but I only specific links for use with Azure SQL and we are using Azure SQL Managed Instances.
If I did not make any misunderstand, in fact, you want to use Managed Identity work with Azure SQL Managed Instance? If this, unfortunately to say, the Managed Identity could not work with Azure SQL Managed Instance. Please see this doc: Services that support managed identities for Azure resources. It list all of the Azure services name which support work with the Managed identities in great detail.
You can see for SQL database, it only support the integration with Azure SQL instead of Azure SQL Managed Instance. That's why you only see the doc link for the usage with Azure SQL.
Until now, the Azure SQL managed instance only support two authentication method:
SQL Authentication:
This authentication method uses a username and password.
Azure Active Directory Authentication:
This authentication method uses identities managed by Azure Active
Directory and is supported for managed and integrated domains. Use
Active Directory authentication (integrated security) whenever
possible.
You can refer to this thread: Managed Identity with Azure SQL Managed Instance?. In this thread, out engineer provided some work around if you trying to configure the app with Managed Identity.
My requirement is, need to launch an AWS EC2 Instance through REST API while TFS running.
For that I need to register the API in TFS as Generic Service Endpoint.
I am not aware how to add the API endpoint because the endpoint contains many header values and signatures.
I hope we can pass the Signature in token.
But what need to pass for UserName?
Could anyone know how to register the AWS REST API in TFS service Endpoint.?
You should do this task via powershell script and just add a powershell step in the TFS build/release workflow. You have flexibility to run it via powershell command line without the TFS web UI. Also the script is portable. It can be invoked in TFS,Jenkins, etc.
It's not able to register aws ec2 instance API in TFS Generic Service Endpoint.
You could use a powershell script to authenticate with AWS API to launch a EC2 instance and invoke the REST API from your Build Pipeline.
Besides, you could use AWS Tools for Microsoft Visual Studio Team Services.
This tool include a new service endpoint type, AWS, to supply AWS credentials to the tasks at runtime.
It will create an AWS Credentials Connection. Select the AWS endpoint type and provide the following parameters:
A name used to refer to the credentials when configuring tasks that
require AWS credentials
AWS Access Key ID
AWS Secret Access Key
After an AWS subscription has be linked to Team Foundation Server, you could use the task deploy to Amazon EC2 with AWS CodeDeploy.
Note: Minimum supported environments
Visual Studio Team Services
Team Foundation Server 2015 Update 3 (or higher)