Upgrade AzureRM Powershell on Hosted 2017 Agent (VSTS - Visual Studio Team Services) - powershell

Question

I am using release management through Visual Studio Teams Services (online). We use Hosted build Agents and I really want to avoid the overhead of managing custom agents.
One item I do need is the AzureRM PowerShell module. Versions up to 5.1.1 are available on the agent but I need 6.0.0.
What I would like to do is use a step in my release process (PowerShell) to aquire version 6.0.0 and use thart instead, however I cant quite get it to work. I have tried a few approaches that have all come unstuck, the current one is:
Write-Output "------------------ Install package provider ------------------"
Find-PackageProvider -Name "NuGet" | Install-PackageProvider -Scope CurrentUser -Force
Write-Output "------------------ Remove Modules ------------------"
Get-Module -ListAvailable | Where-Object {$_.Name -like 'AzureRM*'} | Remove-Module
Write-Output "------------------ Install the AzureRM version we want - 6.0.1! ------------------"
Install-Package AzureRM -RequiredVersion 6.0.1 -Scope CurrentUser -Force
Write-Output "------------------ Import AzureRM 6.0.1 ------------------"
Import-Module AzureRM -RequiredVersion 6.0.1
This all works fine (i.e. does not crash...) but then when I try and use one of the 6.0.1 cmdlets I get an error.
Get-AzureRmADGroup : The Azure PowerShell session has not been
properly initialized. Please import the module and try again.
Any idea of where I am going wrong or alternate strategies I can use to deploy AzureRM 6.0.1 and use it on a hosted agent?

Answer 1

Thanks to Murray for the initial point in the right direction, to show what I was hoping to do wasn't impossible!
I initially tried to do this within the Azure PowerShell task and got pretty far but hit a dead end with AzureRm.Profile as you cannot unload an old version.
The trick was to understanding how the AzureRM VSTS task does it's dependency setup, it effectively takes the "Azure Powershell Version" string in the VSTS UI and uses it to define an additional search path in the PSModules environmental variable i.e. C:\Modules\azurerm_5.1.1.
It will look in that directory, before searching the user profile, then the global modules path.
As soon as it finds the module it performs the Azure login, which will hamper any hope of removing the module after.
So if you instead use a plain powershell task i.e. one that AzureRM isn't loaded into (as Murray also concluded):
Install-PackageProvider -Name NuGet -Force -Scope CurrentUser
Install-Module -Name AzureRM -RequiredVersion 6.2.1 -Force -Scope CurrentUser -AllowClobber
Notably install-module won't be installing to c:\modules like the vsts image generation project.
I seemed to need AllowClobber to get around problems overriding old powershell versions when experimenting, but I suspect I don't need that anymore.
The elegant solution kicks in when next using the Azure PowerShell script.
The preferred powershell version field filled in with 6.2.1 will add C:\Modules\azurerm_6.2.1 to the PSModules path. This doesn't exist, but thankfully because PSModules still includes the user specific modules path, so our 6.2.1 is loaded by itself!
Luckily the AzureRM.Profile from 5.1.1 is forwards compatible enough that the service principal login performed by the Azure Powershell task still works.
Running
Get-Module AzureRm
Get-AzureRmContext
Will output the versions you are hoping for:
Notably if it weren't able to login, I think System.AccessToken could be used (if the option is switched on in the agent phase level).
Diagnostic Techniques if the workaround needs tweaking:
Things that helped greatly, reading through the code that loads in AzureRM in the task:
https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/AzurePowerShellV3/AzurePowerShell.ps1#L80
https://github.com/Microsoft/vsts-tasks/blob/0703b8869041d64db994934bde97de167787ed2e/Tasks/Common/VstsAzureHelpers_/ImportFunctions.ps1
https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/AzurePowerShellV3/Utility.ps1#L18
And also how the VSTS image is generated:
https://github.com/Microsoft/vsts-image-generation/blob/2f57db26dc30ae0f257a3415d26eaa8eea0febf9/images/win/scripts/Installers/Install-AzureModules.ps1
Enabing System.Debug = true as a environment release variable.
Then using VSCode's Log File Highlighter plugin
I'd encourage anyone who's interested to vote in: https://github.com/Microsoft/vsts-image-generation/issues/149
As the AzureRM version at the time of writing is waaaay out of date on the VSTS Hosted 2017 agent.
Unfortunately I can't submit a PR to upgrade it, as it's pulled in via a zip file hosted privately.

Answer 2

I finally figured it out - adding an answer for anyone else that suffers the same.
The key is to login after the AzureRM module is upgraded.
PowerShell code:
Write-Output "------------------ Start: Upgrade AzureRM on build host ------------------"
Write-Output "- - - - - Install package provider"
Install-PackageProvider -Name NuGet -Force -Scope CurrentUser
Write-Output "- - - - - List Modules Before"
Get-Module -ListAvailable| where {$_.Name -Like “*AzureRM*”} | Select Name, Version
Write-Output "- - - - - Remove alll existing AzureRM Modules"
Get-Module -ListAvailable | Where-Object {$_.Name -like '*AzureRM*'} | Remove-Module -Force
Write-Output "- - - - - Install AzureRM 6.0.1"
Install-Module -Name AzureRM -RequiredVersion 6.0.1 -Force -Scope CurrentUser
Write-Output "- - - - - Import AzureRM 6.0.1"
Import-Module AzureRM -Force -Verbose -Scope Local
Write-Output "- - - - - List Modules After"
Get-Module -ListAvailable| where {$_.Name -Like “*AzureRM*”} | Select Name, Version
Write-Output "------------------ End: Upgrade AzureRM on build host ------------------"
Write-Output "------------------ Start: LoginToAzure ------------------"
$SecurePassword = ConvertTo-SecureString $AdminPassword -AsPlainText -Force
$AdminCredential = New-Object System.Management.Automation.PSCredential ($AdminUserEmailAddress, $SecurePassword)
Login-AzureRmAccount -Credential $AdminCredential
Get-AzureRmSubscription –SubscriptionId $SubscriptionId | Select-AzureRmSubscription
Write-Output "------------------ End: LoginToAzure ------------------"