I happened to notice that a data factory (ADFv2) is owner of one of our Azure subscriptions. This is a typical ADF and not doing anything out of ordinary. I am guessing this was done by mistake -- but just want to ask here in the forum if there can be a genuine scenario where data factory needs to be owner of a given subscription?
Yes, what #Nandan said is basically correct, more specifically, essentially the Owner is assigned to the MSI of your ADFv2, when you need to use the MSI(managed identity) of your ADFv2 to auth to Azure resources/call Azure REST APIs, the RBAC role(in your case, it is Owner) is needed, to do most of the things, some other roles are enough e.g. Contributor, to do something like create role assignment, you need the role has the higher privilege, e.g. Owner.
Three specific cases I answered to help you understand:
How to call a REST API using Azure Data Factory Pipelines
How can I use this API in Azure Data Factory
Azure Data Factory pipeline to start SSIS Integration Runtime authorization error
Is the Azure data factory used for any automation purposes using REST APIs, etc ?
Like scaling resources or pausing/resuming resources across RGs?
If that is the case, then the ADF might have been given owner access on subscription for MSI in all the objects within RGs present in the subscription.
If this ADF is specific to data ingestion use cases, then there is no need to provide the ADF owner access on the entire subscription.
Related
I was "david.corbin#companyA.com" now I am "boogie.woogie#companyB.com". Both are AAD backed identities.
The goal is to change how I authenticate to Azure DevOps so that I have one consistent history without "fracturing" or "historical user".
I have tried (in a playpen environment) various items, but I always end up with there being two distinct identities.
In the past during migrations, Microsoft Support was able to alter identities, but I am looking for a solution that does not involve opening a bunch of support tickets.
Azure DevOps Change User Identity
I am afraid there is no such solution that you could resolve this issue by yourself.
If you have already add the new account boogie.xxxx#companyB.com in the organization, we could not map that old david.xxxx#companyA.com to their Azure AD identities.
That is because it will be treated as a new account and assigned a unique VSID when you add the account account boogie.xxxx#companyB.com to organization.
That is also the reason why you always end up with there being two distinct identities.
To resolve this issue, we need to contact the support team to map two account as one in the backend. You could submit the ticket on the Developer Community if you do not want open a support ticket. Or you could create a technical support ticket from Azure Portal if you have Azure support plans.
I want to perform some validation checks in ADF on my input data and any validation failures want to capture into Azure log analytics.
Can someone guide me how to capture the custom logs into log analytics through Azure Data Factory please.
Any example dataflow/pipeline would be very helpful.
Thanks,
Kumar
If I understand correctly you want to be able to get the Azure Monitor logs for ADF and query/store these logs?
Well the good news is most of the information you would want to see is already collected through Azure Monitor.
One of the simple methods to pull the information is to use the Azure Monitor REST API. You can then store the response into a file or table, or you can just query the api for specific pipelines or triggers etc.
Here is a link with example of Authorization and using the Azure Monitor API:
https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/rest-api-walkthrough#authenticating-azure-monitor-requests
This is an example of the HTTP URL using the Azure Monitor REST API to get Activity Run data (Dynamic content syntax):
#{concat('https://management.azure.com/subscriptions/', linkedService().SubscriptionID, '/resourceGroups/', linkedService().ResourceGroupName, '/providers/Microsoft.DataFactory/factories/', linkedService().DataFactoryName, '/pipelineruns/', linkedService().RunID, '/queryActivityruns?api-version=2018-06-01')}
Here are all the different ADF Metrics that can be pulled from Azure Monitor:
https://learn.microsoft.com/en-us/azure/data-factory/monitor-using-azure-monitor#data-factory-metrics
Here is an example of a REST Linked service in ADF that can be used to call the rest api:
You could then create a dataset that will pass all the values to the linked service so that you can call the API, and copy the response to a DB, like here:
This particular example is getting the status of a specific Pipeline RunID, but it can be much more broad than that.
Here is what the request body looks like so that I could filter to just failed pipelines in the last day:
I'm looking into this myself, as far as I can tell you would have to use a REST or HTTP connector to send a POST request to the HTTP Data Collector API in Log Analytics. More details here: https://learn.microsoft.com/en-gb/azure/azure-monitor/logs/data-collector-api
i have two azure account with different subscriptions and i want to move azure factory from one account to another.Any help would be appreciated.
Thanks
You could export the arm template of your ADF account and import it into other destination.Please refer to this document:Continuous integration and delivery in Azure Data Factory.
You could customize the parameters of ARM template to change the different subscription information.
I'm using cloudformation to create one admins group for my aws account, and I need to add a custom tag to all of them in order to save their e-mail for notification purpouse. There's no info in aws docs about it.
Is possible tag an IAM resource? I know that is possible to do it for a EC2 instance.
No, currently no IAM objects can be tagged, in cloudfront or using administration console.
I suppose that since Tags are mostly used for billing and resource grouping, and IAM objects are free of charge and represent authority, roles and permissions and not a resource or a general purpose database.
I think the best alternative for your scenario would be something like DynamoDB to store this configuration.
What I want to Do:- I want to report the status/Health along with percentages of the data replicated for all the servers in 1 ARM portal based Azure Site Recovery Vault. I am referring to the Protected Items via the Azure Disaster Recovery related Recovery Services (and NOT the Backup items). I want to do this programmatically via PowerShell or using any REST API.
Problem:- I am not able to find any API which can provide this data. Are you aware of any way to fetch this data?
What I have already checked:- I have already checked the PowerShell cmdlets here and did not found anything useful: AzureRM.RecoveryServices Module. I have also checked the REST APIs available via the Explorer Tool at https://resources.azure.com/.
Any pointers will be helpful for this.
Well, there's a REST API for that, consult these links:
https://learn.microsoft.com/en-us/rest/api/recoveryservices/protecteditemoperationstatuses
https://learn.microsoft.com/en-us/rest/api/recoveryservices/protecteditemoperationresults
I got the API for the exact operation from MS support. The link to the same is here:
Get specific details about a protected item.
This API give you list of replication protected items in Azure Site Recovery along with Health of each item.