Unable to change active subscription in yaml/Azure PowerShell pipeline - azure-devops

Question

This is kind of continuation of the issue I faced in here
Let me give a backgroud. This is my yaml pipleine:
parameters:
- name: sub_name # name of the subscription; required
type: string
default: false
steps:
- script: echo "Here is subscription name:" ${{ parameters.sub_name }}
- task: AzurePowerShell#5
displayName: 'Launching Main.yml'
inputs:
azurePowerShellVersion: LatestVersion
azureSubscription: My-SPN # This is my almighty Service Principal
ScriptType: 'FilePath'
ScriptPath: '$(System.DefaultWorkingDirectory)/MyPowerShell.ps1'
ScriptArguments: -sub_name ${{ parameters.sub_name }}
and this is my MyPowerShell.ps1 file:
#param ($sub_name)
Get-AzContext -ListAvailable | Where{$_.Name -match $sub_name} | Set-AzContext
$SID=(Get-AzContext).Subscription.id
Write-Output "The active subscription SID is" $SID
No matter what value the $sub_name is given the output of $SID is always the Subscription Id of my service principal - "My-SPN"
How should I set AzContext properly so it changes active subscription?
The same PowerShell script works fine in Azure CLI but not when yaml has got service principal.
I tried to use Set-AzContext -Subscription $sub_name -TenantId 2a1c169e-715a-412b-b526-05da3f8412fa but ended up with following error:
Starting: Launching Main.yml
==============================================================================
Task : Azure PowerShell
Description : Run a PowerShell script within an Azure environment
Version : 5.209.0
Author : Microsoft Corporation
Help : https://aka.ms/azurepowershelltroubleshooting
==============================================================================
Generating script.
========================== Starting Command Output ===========================
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a_temp\adfb7562-7db5-4be6-ae08-dca4664e460c.ps1'"
Added TLS 1.2 in session.
Import-Module -Name C:\Modules\az_7.5.0\Az.Accounts\2.9.1\Az.Accounts.psd1 -Global
WARNING: Both Az and AzureRM modules were detected on this machine. Az and AzureRM modules cannot be imported in the
same session or used in the same script or runbook. If you are running PowerShell in an environment you control you can
use the 'Uninstall-AzureRm' cmdlet to remove all AzureRm modules from your machine. If you are running in Azure
Automation, take care that none of your runbooks import both Az and AzureRM modules. More information can be found
here: https://aka.ms/azps-migration-guide
Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue
Clear-AzContext -Scope Process
Connect-AzAccount -ServicePrincipal -Tenant 2a1c169e-715a-412b-b526-05da3f8412fa -Credential System.Management.Automation.PSCredential -Environment AzureCloud #processScope
Set-AzContext -SubscriptionId 72245732-XXXXXXX -TenantId 2a1c169e-XXXXXXXX
##[error]Please provide a valid tenant or a valid subscription.
##[error]PowerShell exited with code '1'.
Added TLS 1.2 in session.
Finishing: Launching Main.yml
Please help on how to change an active subscription either in a yaml or in powershell file.
Thanks.

Answer 1

AzurePowerShell task uses Service Principal to authenticate and access Azure resources. Therefore, your service principal needs to have subscription owner or contributor permission.

Answer 2

Like #ZiyangLiu-MSFT mentioned above the service principal I was using hasn't had enough rights on the subscription. Changed it to the one which have had proper permission and this resolved the issue.