Why Azure DevOps masks some non secret variables in the log output? - azure-devops

I am logging several guids in the Azure DevOps output, please observe:
So, the *** (secret) *** is my output, because I know the particular field is a secret. However, the ClientId and TenantId are masked by Azure DevOps on its own accord and I do not understand how Azure DevOps decided it should be masked.
These are not build variables, just some Write-Host statements.
We are using on prem Azure DevOps Server 2019.
EDIT 1
Here is the Powershell script that runs:
function Initialize-AzureSqlDB(
[parameter(Mandatory)][ValidateScript({ Test-Path -LiteralPath $_ -PathType Container })]$DBScriptsSource,
[parameter(Mandatory)][string]$SqlServerName,
[Parameter(Mandatory)][string]$SqlServerResourceGroup,
[Parameter(Mandatory)][string]$SubscriptionId,
[Parameter(Mandatory)][string]$ClientId,
[Parameter(Mandatory)][string]$ClientSecret,
[Parameter(Mandatory)][string]$TenantId,
[string]$DBNamePrefix = "${env:UserName}_"
)
{
# Some code that does Write-Host for all the script parameters
...
The script is invoked from a YAML build template. The actual Service principal credentials are found in the ARM_... environment variables. Here is the YAML code:
- powershell: Initialize-AzureSqlDB -DBScriptsSource ${{ parameters.dbScriptLocation }} -DBNamePrefix ${{ parameters.dbNamePrefix }} -TenantId $env:ARM_TENANT_ID -SubscriptionId $env:ARM_SUBSCRIPTION_ID -ClientId $env:ARM_CLIENT_ID -ClientSecret $env:ARM_CLIENT_SECRET -SqlServerName $env:SqlServerName -SqlServerResourceGroup $env:SqlServerResourceGroup
displayName: Initializing Database(s).
condition: eq(variables['RunDBUpgrade'],true)
Now neither ClientId nor TenantId are defined as build variables. Their actual values do come from an exiting Service Connection Endpoint in Azure DevOps, but even there neither are secret.
So, it is unclear to me how Azure DevOps decides to mask them and not SubscriptionId, for instance. I, for my part, do not want either of them to be masked.
Can anyone explain how Azure DevOps decides what to mask here?
EDIT 2
So, I checked our extension and it contains the following code:
Write-Host "##vso[task.setvariable variable=ARM_SUBSCRIPTION_ID;]$($se.data.subscriptionId)"
Write-Host "##vso[task.setvariable variable=ARM_SUBSCRIPTION_NAME;]$($se.data.subscriptionName)"
Write-Host "##vso[task.setvariable variable=ARM_TENANT_ID;]$($se.auth.parameters.tenantid)"
Write-Host "##vso[task.setvariable variable=ARM_CLIENT_ID;isSecret=true]$($se.auth.parameters.serviceprincipalid)"
Write-Host "##vso[task.setvariable variable=ARM_CLIENT_SECRET;isSecret=true]$($se.auth.parameters.serviceprincipalkey)"
That can explain how ClientId gets masked, but not the TenantId. Now there is a possibility that the code in the master branch does not exactly match the one installed in server. I will make sure the task is redeployed and then we will know.

Related

how can i output current value of a variable in terraform during azure pipeline run?

In powershell i usually use write-host to print out a variable whilst working in an azuredevops pipeline to check the value of a variable.
Can you do this in terraform? I am running some terraform code but want to check some of the terraform variable values during the pipeline run , is there a way of doing this?
According to documentation :
- pwsh: |
Write-Host "Non-secrets automatically mapped in, sauce is $env:SAUCE"
Write-Host "Secrets are not automatically mapped in, secretSauce is $env:SECRETSAUCE"
Write-Host "You can use macro replacement to get secrets, and they'll be masked in the log: $(secretSauce)"
Write-Host "Future jobs can also see $env:SETVARS_OUTPUTSAUCE"
write-Host "Future jobs can also see $(SetVars.outputSauce)"
Just run terraform plan part of the pipeline to check if the variables are set as expected.
You can also reference variables in terraform output if you need to.

Is it possible to get the pool name that is executing an Azure DevOps Pipeline from within the yaml?

As the title says, is there any way of retrieving the pool name in an azure devops pipeline for use in a script to perform conditional actions based on which pool is used for the pipeline?
I have checked the predefined variables and the list of environment variables in the execution but I cant see the pool name itself specified anywhere.
Any help appreciated.
is there any way of retrieving the pool name in an azure devops pipeline for use in a script to perform conditional actions based on which pool is used for the pipeline?
You could use the Agent variables variable Agent.Name to get the pool name in an azure devops pipeline.
And then you could pass this variable to the script to perform conditional actions, like powershell:
For example:
Variable: password (secret)
task: PowerShell
Arguments: -pass $(password)
Script:
Param(
[String]$pass
)
if ($pass) { Write-Host "variable is NOT null" }
if (!$pass) { Write-Host "variable is null" }

AzureDevOps Powershell task command line parameters are not properly rendered in Powershell script

I have an AzureDevops Powershell task which is passing values read from KeyVault as parameters to the script. Even with double or single quotes surrounding the variables in the DevOps task, the values do not appear correct in the script.
What else do I need to do to get these values passed properly to the script?
There is an older similar question and the answer was to double-quote the parameters in the task, which I've done, to no avail.
Sample Powershell task:
ScriptArguments: -connectionString "$(ConnString)" -password "$(ConnStringPassword)"
Sample values stored in KeyVault:
ConnString: Server=tcp:test-sqldb.database.windows.net,1433;Initial Catalog=TestDb;Persist Security Info=False;
ConnStringPassword: Pa$$w0rd
Sample Script:
param (
[string]$connectionString,
[string]$password
)
Write-Host "connectionString: $connectionString"
Write-Host "password: $password"
Sample output from pipeline execution:
connectionString: ***
password: Paw0rd
If I use single quotes in the pipeline task as suggested by #Thomas below:
Sample Powershell task:
ScriptArguments: -connectionString '$(ConnString)' -password '$(ConnStringPassword)'
Sample output from Pipeline execution:
connectionString: ***
password: ***
#ShaykiAbramczyk is correct, secrets are not viewable. The values are however being properly utilized in the Powershell script. I verified this by publishing a test output file and seeing that the KeyVault values are in the output file. The following reference gave me the idea to publish to see the values as opposed to trying to echo or write them to the console.
Use secrets from Azure Key Vault in Azure Pipelines

Use Azure DevOps service principal details in Azure Powershell task

When writing an Azure CLI script in a Azure DevOps pipeline you can get access to the the serviceprincpal id, key and tenantid. Is there a way to get this info in a Azure Powershell task?
The addSpnToEnvironment input which adds service principal id and key of the Azure endpoint you chose to the script's context is one option available only in Azure ClI Task, but not Azure Powershell Task.
Is there a way to get this info in a Azure Powershell task?
As an alternative workaround, we can define job-scoped variables in Azure ClI Task, check this document.
Steps to test:
1.Using latest Azure CLI task 2.0-preview and choose Powershell type. Try inline script like this:
Write-Host "##vso[task.setvariable variable=SpId;]$env:servicePrincipalId"
Write-Host "##vso[task.setvariable variable=SpKey;]$env:servicePrincipalKey"
Write-Host "##vso[task.setvariable variable=TenantId;]$env:tenantId"
Write-Host "##vso[task.setvariable variable=TestVar;]JustForTest"
2.Then add one Azure Powershell task after Azure CLI Task to test:
Write-Host $env:SpId
Write-Host $env:SpKey
Write-Host $env:TenantId
Write-Host $env:TestVar
3.Output:
So if you define the job-scoped variables using Write-Host "##vso[task.setvariable variable=xxx]xxx"(Powershell) or echo "##vso[task.setvariable variable=xxx]xxx"(Batch), you can then use something like $env:VariableName to access the info. The *** in log is because these are secrets projected by Azure Devops, so they're masked.
You can invoke a powershell script via Azure CLI task and use the 'addSpnToEnvironment' flag as explained in accepted answer.
- task: AzureCLI#2
displayName: 'Custom ps script'
inputs:
azureSubscription: ${{ parameters.serviceConnection }}
addSpnToEnvironment: true # Adds DevOps SP details to context
scriptType: pscore
scriptPath: '$(System.DefaultWorkingDirectory)/somescript.ps1'

Use Azure CLI within Azure Powershell Task

I want to create a Powershell script which executes some AzureRm... commands and follows those up with some Az commands. Reason being that some commands are only available via Az.
When trying to execute these scripts in a release pipeline, the script always fails with the following error:
ERROR: Please run 'az login' to setup account.
Executing the Az commands in a Azure CLI task work as expected, because Az Login is executed by the task.
I don't want to pass the secret required to login to the script if at all possible. I would rather fall back to separating the scripts into two steps in the pipeline.
Is it possible to use the Azcommands within a Azure Powershell task without passing the secrets manually?
Minimal example:
Create a new release pipeline
Add a task Azure PowerShell
Use inline script
As script, execute az account show
The short term solution I already had in place was passing the ServicePrincipal information into the powershell script and executing az login manually (same as Bevan's answer below).
My long term solution was to replace all Azure CLI calls with "Az Powershell" commands.
Luckily, most commands are available by now.
A couple of commands don't have an equivalent commandlet. But if they are available via ARM, you can figure out an alternative command with Powershell.
Many of them involve using New-AzResource/New-AzureRmResource or Invoke-AzResourceAction/Invoke-AzureRmResourceAction
How to create a Linux AppService Plan with New-AzAppServicePlan?
Manage Azure Cosmos DB SQL API resources using PowerShell
# AzureCLI
az cosmosdb list-keys
# Powershell:
$keys = Invoke-AzResourceAction -Action listKeys `
-ResourceType "Microsoft.DocumentDb/databaseAccounts" -ApiVersion "2015-04-08" `
-ResourceGroupName $resourceGroupName -Name $accountName
When I have mixed commands I put this into my Azure Powershell task
az login --service-principal --username "$(ServicePrincipal)" --password "$(AzureDevOps-ServicePrincipal-Secret)" --tenant "$(Azure_Tenant)"
I have my SP and Tenant IDs as a variables and the Secret for the SP stored in Azure KeyVault linked to a Library Variable group. You can alternatively just stored the secret in a normal Variable/Variable Group and hit the padlock icon to secure it.
You may need to run az account set -s $(SubscriptionName) if the SP has access to multiple subscriptions in the same tenant.
I figured out this approach - store credentials in job scoped variables (currently only an Azure CLI task allows that) and then re-use in Azure PowerShell task:
- task: AzureCLI#2
displayName: 'Azure CLI - get credentials'
inputs:
azureSubscription: 'SUBSCRIPTIONNAME'
scriptType: 'pscore'
scriptLocation: 'inlineScript'
addSpnToEnvironment: true
inlineScript: |
Write-Host "##vso[task.setvariable variable=ARM_CLIENT_ID]$($env:servicePrincipalId)"
Write-Host "##vso[task.setvariable variable=ARM_CLIENT_SECRET]$($env:servicePrincipalKey)"
Write-Host "##vso[task.setvariable variable=ARM_TENANT_ID]$($env:tenantId)"
- task: AzurePowerShell#5
displayName: 'collector'
inputs:
azurePowerShellVersion: LatestVersion
azureSubscription: 'SUBSCRIPTIONNAME'
pwsh: true
scriptType: inlineScript
inline: |
az login --service-principal --username "$($env:ARM_CLIENT_ID)" --password "$($env:ARM_CLIENT_SECRET)" --tenant "$($env:ARM_TENANT_ID)"
./mixedscript.ps1
Anyway, it wont work like that, because you have to authenticate to az utility separately. az cli and powershell do not share connection information. you can try and use az step with some command before powershell step. that would force az to auth and after that you can use it inside powershell ste.