How to sign in interactively as a Service Principal to access Azure Repo? - azure-devops

I have Azure Devops pipelines working nicely (YAML not GUI).
The Service Principal has a secret that I have created (and stored in a Key Vault) to allow the pipeline to log in during the pipeline for 2 things that require REST API calls (az rest) and these require a "manual" login. This is working perfectly.
However, I now need the same Service Principal to read the commmit titles during the pipeline (the code and deployments are different projects). From what I have read, this needs more REST API calls (az rest). So, I have created the REST API calls in the CLI and run this under my account in the Cloud Console. This also works (after I have done an "az login" and logged in as me).
All I now need to do is repeat this under the context of the Service Principal (which already has permissions to the repo via the Project Administrators group - I will trim this down later once it works). So, I login as the SP and get a Bearer token. This all works fine.
The problem occurs when I call the REST API to access the commit details. I do an az rest to this URI:
az login --service-principal -u "<guid>" -p "<password>" --tenant "<guid>"
$request = "https://dev.azure.com/<org>/<project>/_apis/git/repositories/<repo>/commits/<commit-id>"
$accessToken = az account get-access-token --query "accessToken" --output tsv
$token = "Bearer $accessToken"
$headers = #{ Authorization = $token }
$comment = az rest --method get --headers ( $headers | ConvertTo-Json ) --uri $request --query "comment" --output tsv
I get an error:
Please sign-in at least once as \\ in a web browser to enable access to the service
We know that the permissions are therefore working. However, my question is:
How am I supposed to log in interactively as the service principal in a web browser?
Is there a better way to retrieve the commit title from the repo? (In other words, am I disappearing down a rabbit hole?)

Instead of using a service principal, use the oauth token assigned to the job in Azure Pipelines.
steps:
- powershell: |
Write-Host "This is a script that could use $env:SYSTEM_ACCESSTOKEN"
Write-Host "$env:SYSTEM_ACCESSTOKEN = $(System.AccessToken)"
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken
You may need to assign additional permissions.
https://learn.microsoft.com/en-us/azure/devops/pipelines/policies/permissions?view=azure-devops
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/access-tokens?view=azure-devops&tabs=yaml#manage-build-service-account-permissions

Related

How to use Service Principal Authentication (Bearer Token) in Powershell for DevOps without being asked to sign in?

I'm trying to use Powershell to create a bug in DevOps using Service Principal authentication (Bearer token). Using my Personal Access Token I'm able to do it. Using the Bearer token it is asking me to sign in. How can I use the Bearer token without being asked to sign in? I need the whole process to be automated, no interaction to sign in.
I'm successfully getting the Bearer token this way:
$AppID = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
$Secret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
$TenantID = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
$Resource = "https://management.azure.com/"
$TokenUri = "https://login.microsoftonline.com/$TenantID/oauth2/token/"
$Body = "client_id=$AppId&client_secret=$Secret&resource=$Resource&grant_type=client_credentials"
$TokenResult = Invoke-RestMethod -Uri $TokenUri -Body $Body -Method "POST"
$AccesToken = $TokenResult.access_token
Is there a setting in the Azure Active Directory setup I'm missing? Maybe something in the Authentication section or API Permissions? Thanks!
Yes, as mentioned in the doc,
The Azure DevOps API doesn't support non-interactive service access via service principals.
which means you could not Azure AD client credential flow get the token to call the DevOps API(the script you provided uses this flow), as there is no access control of service principal in Azure DevOps. The supported are the user-involved ways, e.g. auth code flow.
In this case, if you want to use a non-interactive way to call the DevOps REST API via powershell, there are two workarounds.
Use the Azure AD ROPC flow in powershell, it is not recommended, because we need to expose the username and password in the request, it is not secure, and it will not work for MFA-enabled accounts.
Use the PAT, just base64 encoded the token, then call the REST API with Invoke-RestMethod it is a recommended way.

Identity Governance with Microsoft Graph results in 403 Unknown Error (PIM)

There is a feature in Azure which is called Identity Governance or Entitlement Management. This feature allows to create access packages and manage user permissions with a request-approve workflow.
I want to automate the creation of the AccessPackages and AccessPackageCatalog Resources. Azure CLI and the Azure PWSH Module do not support the AccessPackages. The only way to automate this is described here: https://github.com/MicrosoftDocs/azure-docs/issues/52179 which states: "Use the Microsoft Graph beta API"
I tried the following to access the API from the commandline:
Using plain powershell:
$tokenJson=$(az account get-access-token --resource-type ms-graph)
$tokenObject=$(echo $tokenJson | ConvertFrom-Json)
$headers = #{Authorization = "Bearer $($tokenObject.accessToken)"}
$result = Invoke-WebRequest https://graph.microsoft.com/beta/identityGovernance/entitlementManagement/accessPackages -Headers $headers
or using az rest
az rest --method get --uri https://graph.microsoft.com/beta/identityGovernance/entitlementManagement/accessPackages
both request result in the following error with a 403 response:
{
"error": {
"code": "UnknownError",
"message": "",
"innerError": {
"request-id": "0cd50759-d95a-4171-a942-3424cb19a622",
"date": "2020-05-26T15:34:48"
}
}
}
When I try to view the AccessPackages inside the Portal everything works as expected: I can list and create the Packages.
SIDE NOTE I do NOT have permanent access to the Identity Governance pane in azure. Instead I must use Privileged Identity Management to Activate the User Administrator Role for my Account.
az logout && az login after Role activiation did not change anything.

Having no permission for updating Variable Group via Azure DevOps REST API from running Pipeline

In a nutshell, I am trying to update a Variable Group when executing a PowerShell script from a build Pipeline (increasing a build number, but this part is not so important).
As it suggested in this topic and Azure DevOps docs I use code similar to:
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build-release/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=2.0"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 1000)"
Everything works fine in general except one thing. I seem don't have some sort of permission to update this value receiving the following error message:
You do not have permissions to perform this operation on the variable
group. A variable group Administrator should add you to the
Administrator role.
However:
It's a bit weird as $env:SYSTEM_ACCESSTOKEN to me should actually belong to the process (the Pipeline executing process) which is invoked from inside Azure, so it also should have access to Variable Groups. (In any case, I am not aware of what's the Identity of the currently running Pipeline and where it's setup).
I can understand that the process is executed on one machine (actually, across different systems, so that's all complicated and doesn't mean the Pipeline executing Identity automatically has the access to Variable Group directly), but I can't find any documentation on how to setup all those access rights. (The articles on the links attached mention nothing about the access rights).
I've tried to add/setup any (possibly related) users/groups to the Group Variable Permissions (Security section of the Variable Group section), but still no luck with the REST API updates.
So, any hints on setting up the permissions for currently executing Identity of build Pipeline?
The OAuth Access Token is created with "Project Collection Build Service(xxx)" account, but not your account.
So you just need to add "Project Collection Build Service(xxx)" account as Administrator role for the variable group.
For anyone running into this error today. It seems MS has changed some things. If you pay close attention to the error message, it now tells you what account needs access. Add this account to the permissions for the variable group library. In my case it was an account named "(My Organization) Platform Build Service"

Add certificate from Azure Key Vault to Azure App Service via REST API

I'm trying to add a private self-signed certificate to an Azure App Service (as in the screenshot at the bottom) via the REST API (in PowerShell). I call the API as follows:
$certBody = #{
name = "InfoServiceTAKeyVaultDev"
location = "West Europe"
properties = #{
keyVaultId = "/subscriptions/<subscriptionId>/resourceGroups/BzInfoServiceTADEV/providers/Microsoft.KeyVault/vaults/BzKVInfoServiceTADev"
keyVaultSecretName = "InfoServiceTAKeyVaultCert"
}
}
Invoke-RestMethod `
-Method Put `
-Uri ("https://management.azure.com/subscriptions/<subscriptionId>" +
"/resourceGroups/BzInformatieServiceResourceGroupDEV" +
"/providers/Microsoft.Web/certificates" +
"/InfoServiceTAKeyVaultDev" +
"?api-version=2016-03-01") `
-Headers #{ Authorization = ("Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSU...")
"Content-Type" = "application/json" } `
-Body ($certBody | ConvertTo-Json -Compress -Depth 3)
The result is an error message: The service does not have access to
'/subscriptions/<subscriptionId>/resourcegroups/bzinfoservicetadev/providers/microsoft.keyvault/vaults/bzkvinfoservicetadev' Key Vault. Please make sure that you
have granted necessary permissions to the service to perform the request operation.
In this context, who is 'the service' that does not have access to this Key Vault? I already found some posts claiming I should add service principal abfa0a7c-a6b6-4736-8310-5855508787cd to my Key Vault access policies but that does not have any effect.
Screenshot of the desired end result:
UPDATE: I had already enabled the advanced access policy Enable access to Azure Resource Manager for template deployment. This also does not do the trick.
The comment by #tim-scriv proved to be very helpful. I had to add the following principal to the Key Vault access policies: Microsoft Azure App Service (object id: 3ed082e0-d6c4-4071-ac9e-61363535a0a3). Permission to get secrets is enough.
You will need to add the Enable access to Azure Resource Manager for template deployment advanced access policy.
You are not directly adding the cert to the webapp, you are telling the webapp where the cert exists. The web app relies on Resource Manager to actually go and collect the certificate and install it into the webapp.
Key vault does have extensive audit logging that can be enabled to understand what is attempting to access secrets that makes these issues a little easier to debug.
I had the above issue and fixed it by following this article https://github.com/Azure/azure-quickstart-templates/tree/master/201-web-app-certificate-from-key-vault;
By default, 'Microsoft.Azure.WebSites' Resource Provider (RP) doesn't have access to the Key Vault specified in the template hence you need to authorize it by executing the following PowerShell commands before deploying the template:
Login-AzureRmAccount
Set-AzureRmContext -SubscriptionId AZURE_SUBSCRIPTION_ID
Set-AzureRmKeyVaultAccessPolicy -VaultName KEY_VAULT_NAME -ServicePrincipalName abfa0a7c-a6b6-4736-8310-5855508787cd -PermissionsToSecrets get
ServicePrincipalName parameter represents Microsoft.Azure.WebSites RP in user tenant and will remain same for all Azure subscriptions. This is a onetime operation. Once you have a configured a Key Vault properly, you can use it for deploying as many certificates as you want without executing these PowerShell commands again.
Adding the Microsoft Azure App Service principal to have GET access to the KeyVault also fixed the same issue for me.
In my case I was trying to execute an ARM Template through Terraform using the azurerm_template_deployment command to add an SSL Cert to an Azure Web App Service and then bind the cert to the URL.

How to use Azure AD Graph API access for service principals?

I have a working Azure AD/Azure daemon application using adal4j that uses user/password authentication. Due to issues with ADFS, I wish to also be able to authenticate using a service principal (client ID/secret). This seems to work fine for the Azure (non-AD) portion of the app, as the SP roles can be defined for the subscriptions in question, however for the Azure AD part, I get:
response code 403, error: Authorization_RequestDenied: Insufficient privileges to complete the operation.
...this occurs on the first call to the Graph API - I get valid tokens from AuthenticationContext.acquireToken() using the https://graph.windows.net scope.
My account is an Owner in the directory. I've tried using the "Grant Permissions" button on the app, and have also tried fabricating a consent URL (which works) and using that to consent to the app having the necessary privileges in the directory. Neither seems to affect this.
The app is a Native app, as it is a daemon/service app, so can't participate in OAuth2 consent.
How does one access the Azure AD Graph API using a SP to authenticate? As a reminder, unchanged, the app works with (non-ADFS) user/password, and the SP works with the Azure API, just not Azure AD Graph API.
Thanks...
P.S. I've also tried this with the Azure Graph API, which Microsoft now recommends instead of the Azure AD Graph API. Same result, and similarly works with user/password creds.
Amending this to kind of take adal4j out of the picture - this seems to be more of a generic Azure AD problem. Here's an example of the problem, using curl:
Client credentials token request:
curl --request POST "https://login.windows.net/367cxxxx41e5/oauth2/token" --data-urlencode "resource=https://graph.windows.net" --data-urlencode "client_id=9d83yyyy08cd" --data-urlencode "grant_type=client_credentials" --data-urlencode "client_secret=secret"
Client credentials token response:
{"token_type":"Bearer","expires_in":"3599","ext_expires_in":"0","expires_on":"1491486990","not_before":"1491483090","resource":"https://graph.windows.net","access_token":"eyJ0zzzz2zVA"}
Azure AD REST query using client credentials token:
curl "https://graph.windows.net/367cxxxx41e5/tenantDetails" --header "Authorization: eyJ0xxxx2zVA" --header "Accept: application/json;odata=minimalmetadata" --header "api-version: 1.6"
Azure AD REST response using client credentials token:
{"odata.error":{"code":"Authorization_RequestDenied","message":{"lang":"en","value":"Insufficient privileges to complete the operation."}}}
Now, contrast that with (note the same tenant ID/app ID):
Password credentials token request:
curl --request POST "https://login.windows.net/367cxxxx41e5/oauth2/token" --data-urlencode "resource=https://graph.windows.net" --data-urlencode "client_id=9d83yyyy08cd" --data-urlencode "grant_type=password" --data-urlencode "username=bozo#clown.com" --data-urlencode "password=password"
Password credentials token response:
{"token_type":"Bearer","scope":"Directory.AccessAsUser.All Directory.Read.All Group.Read.All Member.Read.Hidden User.Read User.Read.All User.ReadBasic.All","expires_in":"3599","ext_expires_in":"0","expires_on":"1491489157","not_before":"1491485257","resource":"https://graph.windows.net","access_token":"eyJ0zzzz0EXQ","refresh_token":"AQABzzzzAgAA"}
Azure AD REST query using password credentials token:
curl "https://graph.windows.net/367cxxxx41e5/tenantDetails" --header "Authorization: eyJ0xxxx0EXQ" --header "Accept: application/json;odata=minimalmetadata" --header "api-version: 1.6"
Azure AD REST response using password credentials token:
{"odata.metadata":"https://graph.windows.net/367cxxxx41e5/$metadata#directoryObjects/Microsoft.DirectoryServices.TenantDetail","value":[{"odata.type":"Microsoft.DirectoryServices.TenantDetail","objectType":"Company","objectId":"367cxxxx41e5","deletionTimestamp":null,"assignedPlans":[{"assignedTimestamp":"2017-02-24T03:25:33Z","capabilityStatus":"Enabled","service":"SharePoint","servicePlanId":"e95byyyyc014"},...
My suspicion at this point is that the SP, created by default by Azure AD when the app was created, doesn't include the necessary permissions. I'll try creating a new SP with the rights specified. Examples of this abound, but all are focused on the target app being some mythical LOB app, not Azure AD. And, as seems usual with the half-baked portal, the CLI must be used to do this.
</snark>
Also, I've verified that anyone with Reader role should be able to execute that query. I've added both Contributer and Owner to the SP, with no effect.
Also also, FWIW, I've verified that the SP has, in theory, the Azure AD (and other) permissions I entered in the portal. I think.
PS C:\> Get-AzureADServicePrincipalOAuth2PermissionGrant -objectid 'a9f9xxxx5377'|format-list -property consenttype,resourceid,scope
ConsentType : AllPrincipals
ResourceId : c569xxxxe7f0
Scope : Member.Read.Hidden User.Read User.ReadBasic.All User.Read.All Group.Read.All Directory.Read.All
Directory.AccessAsUser.All
ConsentType : AllPrincipals
ResourceId : 3318xxxx66a5
Scope : user_impersonation
ConsentType : AllPrincipals
ResourceId : 8c0fxxxx4198
Scope : User.Read.All User.ReadBasic.All Group.Read.All Directory.Read.All Directory.AccessAsUser.All User.Read
PS C:\> get-azureadobjectbyobjectid -objectids 'c569xxxxe7f0','3318xxxx66a5','8c0fxxxx4198'
ObjectId AppId DisplayName
-------- ----- -----------
8c0fxxxx4198 00000003-0000-0000-c000-000000000000 Microsoft Graph
3318xxxx66a5 797f4846-ba00-4fd7-ba43-dac1f8f63013 Windows Azure Service Management API
c569xxxxe7f0 00000002-0000-0000-c000-000000000000 Microsoft.Azure.ActiveDirectory
According to your description, per my experience, I think the issue was caused by two reasons.
Not add api access of Windows Azure Active Directory (Microsoft.Azure.ActiveDirectory) in the Required permissions tab of your registed application in Azure AD on Azure portal as below and select the related permissions.
As references, you can refer to the other SO thread Trouble with authorization using client_credentials Azure AD Graph API or the offical document here and a helpful blog.
Not assign Contributor for this service principal. You need to run the powershell command below to do this.
New-AzureRmRoleAssignment -RoleDefinitionName Contributor -ServicePrincipalName 'applicationID'
Or you can also refer to my answer for another SO thread Cannot list image publishers from Azure java SDK to do this via Azure CLI or just on Azure portal.
Hope it helps.