How do I automatically create service principals or MSIs with Terraform for use in Azure Pipelines to manage AKS resources? - azure-devops

Question

I'm following the official docs to create Azure Kubernetes clusters. The docs state that I need to create a service principal first, manually, and provide the client_id and client_secret.
Doing it manually is not an option.
Here is the code for my service principal. It's decorated with links to the most recent Terraform docs for reference.
data "azurerm_subscription" "current" {}
data "azuread_client_config" "current" {}
resource "random_id" "current" {
byte_length = 8
prefix = "ExternalDnsTf"
}
# Create Azure AD App.
# https://registry.terraform.io/providers/hashicorp/azuread/latest/docs/resources/application
resource "azuread_application" "current" {
display_name = random_id.current.hex
owners = [data.azuread_client_config.current.object_id]
}
# Create Service Principal associated with the Azure AD App
# https://registry.terraform.io/providers/hashicorp/azuread/latest/docs/resources/service_principal
resource "azuread_service_principal" "current" {
application_id = azuread_application.current.application_id
app_role_assignment_required = false
owners = [data.azuread_client_config.current.object_id]
}
# Create Service Principal password
# https://registry.terraform.io/providers/hashicorp/azuread/latest/docs/resources/application_password
resource "azuread_application_password" "current" {
application_object_id = azuread_application.current.object_id
}
# Create role assignment for service principal
# https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment
resource "azurerm_role_assignment" "current" {
scope = data.azurerm_subscription.current.id
role_definition_name = "Contributor"
# When assigning to a SP, use the object_id, not the appId
# see: https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-cli
principal_id = azuread_service_principal.current.object_id
}
I keep getting the following error in my pipeline: (note, I am the owner of my subscription)
ApplicationsClient.BaseClient.Post(): unexpected status 403 with OData
│ error: Authorization_RequestDenied: Insufficient privileges to complete the
│ operation.
What I'm trying to do is to eliminate the manual steps to setup supporting services. Take ExternalDNS for example. The Azure docs state that I need to use az ad sp create-for-rbac -n ExternalDnsServicePrincipal; az role assignment create --role "Reader" --assignee <appId GUID> --scope <resource group resource id>; az role assignment create --role "Contributor" --assignee <appId GUID> --scope <dns zone resource id>
Ultimately, I'm trying to create the terraform version of the azure cli commands.
Support for create-for-rbac was a feature request on github. That used to work great, but so much has changed, it's not applicable to current API versions. Also, with AAD Graph being deprecated in favor Microsoft Graph API, I wonder if I'm getting snagged on that.
The ExternalDNS docs also suggested Managed Service Identities (MSI). Service principals, MSI, MSGraph API integration, honestly, I don't care which one is used. Whatever is current best-practices is fine so long as I do not have to log into the portal to manually create or give permissions, or manually run az cli commands.
EDIT: Permissions clarification
I'm using Terraform, of course, to provision resources. If I do all of this without terraform (manually or with a bash script), I use azure cli I start setting permissions by doing the following:
az login
az account set -s <my-subscription-id>
I am the owner of my subscription. I can run all commands, create SPs, MSIs, assign roles, etc, with no problem.
In the pipelines, I am using the charleszipp az pipelines terraform plugin. In the logs, I see:
az login --service-principal -t <my-tenant-id> -u *** -p ***
az account set -s <my-subscription-id>
I'm not sure if that makes a difference. I interpret that as ultimately, commands are executed after signing in and setting the account subscription, like I do manually.
Technically, I'm not using a service connection in several of these tasks. However, where one is required, I have created a Service connection and defined its Scope to the subscription level. It's of type Azure Resource Manager.
However, if I click "manage service principal, it takes me to the portal where there are no permissions defined.
While I am the owner of my subscription, I am not the root management group. I'm owned / provisioned by someone else. Ultimately, they have control of Active Directory. I cannot add or edit permissions. If I try to add any under permissions API and select Microsoft Graph, it says that authorization is required. Grant Admin Consent for <parent organization is greyed out.
But why would that be important if I'm the owner of my subscription? If I can do whatever I want via the az cli command line, what's preventing me from doing the same in the pipeline?

Answer 1

I am using user-managed identity for that, it seemed most straightforward and worked fine for me.
resource "azurerm_user_managed_identity", "mi" {
resource_group_name = "rg"
name = "mi"
location = "eastus"
}
resource "azurerm_role_assignment" "ra" {
scope = azurerm_subnet.sn.id // subnet I created earlier
role_definition_name = "Network Contributor" // required with kubenet
principal_id = azurerm_user_managed_identity.mi.principal_id
}
resource "azurerm_kubernetes_cluster" "aks" {
name = "aks"
identity {
type = "UserAssigned"
user_assigned_identity_id = azurerm_user_managed_identity.mi.id
}
<...remaining attributes...>
depends_on = [azurerm_role_assignment.ra] // just to be safe
}