Retrieve Azure DevOps pipeline variable using PowerShell variable - powershell

I'm using Azure DevOps Release pipeline which has next steps:
Download KeyVault secrets
Invoke Console App with correct parameters
Downloading of KeyVault secrets works fine and I can confirm that they are available by using $(Key1) where Key1 is actual key stored in KeyVault secrets.
Now, what I want is to loop against list of the secrets (simple text file with keys separated by comma) and append them to a console app parameter, but I fail to retrieve Azure DevOps pipeline variable with PowerShell variable.
$keyVaultVariables can be Key1,Key2,Key3 which corresponds to the keys stored in KeyVault, meaning that when I'm calling $($kvVar) is should get value of the secret with the key. What I get is just key, but no value.
$keyVaultList = $keyVaultVariables -split ','
$stringReplacementValues = ""
foreach($kvVar in $keyVaultList)
{
$val = $($kvVar)
Write-Host $val
$stringReplacementValues = $stringReplacementValues + "$kvVar|$val;"
}
Write-Host $stringReplacementValues
What am I doing wrong?

What you want cannot be done this way. It's a security feature.
Secrets can only be iterated through the task-sdk from a custom task. Any script or existing task that doesn't have this functionality needs to have these values passed in through an input or the environment or through inlining the value in the script directly. This is a security feature to prevent say a roque npm package from extracting all of the secrets from a pipeline.
If you move your functionality into a custom task, it could access the secrets.
The corresponding PowerShell function.
The corresponding Typescript Function

Related

Cannot read secret variable from AWS SSM Get Parameter task in Azure DevOps Pipeline

My Azure DevOps pipeline has a "AWS SSM Get Parameter" task that reads the parameters into build variables. I can reference these variables successfully using "$(BuildVariableName)" syntax, as long as they are not secret parameters, but for any parameter where "is secret = true", the value of the variable is "***". What do I need to do to be able to read the secret parameters as build variables?
This might not be the most desirable solution, but I was able to partially get this to work by using a Powershell task to copy the secure variable to a non-secure variable.
Write-Host "##vso[task.setvariable variable=NonSecureVariable;]$(SecureVariable)"
However, if $(SecureVariable) contains "$", then all of the characters after the "$" are truncated. Is there a way to get access to the secure variable other than Powershell, or is there a way to get Powershell to work with special characters?

How to use Terraform output variables stored in an storage account in Azure DevOps release pipelines?

There are similar questions to this one on Stackoverflow, but none of them addressing my issue in using Terraform when an Azure Storage account is used to retain outputs.
Here is my scenario, which may sound familiar:
My terraform script provisions an Azure HTTP-triggered function with a function key. Also, this terraform script provisions a Web App that calls the mentioned HTTP-triggered function. The HTTP-triggered function's key is stored in the Web App's appsettings.json file to include it in the HTTP headers' calls to the HTTP-triggered function.
The following code snippet illustrates how the HTTP-triggered function is provisioned in terraform:
resource "azurerm_function_app" "myhttpfunc" {
name = var.funcname
location = "${azurerm_resource_group.rc.location}"
resource_group_name = "${azurerm_resource_group.rc.name}"
app_service_plan_id = "${azurerm_app_service_plan.funcsserviceplan.id}"
storage_account_name = "${azurerm_storage_account.funcstorage.name}"
storage_account_access_key = "${azurerm_storage_account.funcstorage.primary_access_key}"
}
The output variables to access the function's keys is per below:
output "funchostkeys" {
value = data.azurerm_function_app_host_keys.myhttpfunc
sensitive = true
}
I assume that this output and others will appear in terraform.tfstate hosted on the dedicated Azure Storage account at some point down the road.
My question is that how one can get that particular output in an Azure Release pipeline to manipulate the appsettings.json file by replacing the configuration entry with what terraform has produced in the output variable?
For example this post suggests producing terraform's output to a file per the following line but that does not seem to be an option in my case because my terraform script leverages Azure storage to retain outputs.
terraform output -json > outputs.json
Based on your requirement, you could try to output the variable with the following command:
terraform output -raw funchostkeys
Then you could use the logging command to set the value as pipeline variable.
For example: Powershell Script
- powershell: |
echo ##vso[task.setvariable variable=varname]$(terraform output -raw funchostkeys)
displayName: 'PowerShell Script'
Then you could use the Replace token task to use the pipeline variable to replace the value in appsettings.json.
You could define #{variablename}# in appsettings.json. When you run the pipeline, the output value will be set in the target place.

Azure DevOps Release Pipelines - Using env parms with a period . in

I am finding using AZDO Release pipeline variables maddening in Powershell steps.
I am running an Azure PowerShell step to return a primary key value. It is 2 lines…
$primarykey = (Get-AzRelayKey -ResourceGroupName ${env:az-resourcegroupname} -Namespace ${env:az-relaynamespace} -HybridConnection ${env:serviceBus.primaryRelay.ConnectionName} -Name ${env:serviceBus.primaryRelay.KeyName} | Select-Object -ExpandProperty PrimaryKey)
Write-Host "##vso[task.setvariable variable=serviceBus.primaryRelay.Key]$primarykey"
In my pipeline I have a mix of variable names, some I have complete control over (the az- prefixed ones) and others I don’t (the ones starting serviceBus.)
The reason I have no control over the latter is that they are used for a later File Transform step that navigates an appsettings.json file to find/replace values, and its unable to be changed (for example serviceBus.primaryRelay.ConnectionName is a value that is changed in the JSON and the file transform step specifies to navigate the JSON structure, it has to be separated with a period . )
When this script runs it always complains about the -HybridConnection value being empty. This is because the variable has a period in it.
I’ve tried everything I can think of to retrieve that value in the code.
Are they suggesting here that a variable with a period isn’t workable in Powershell in AZDO release pipelines? I’m completely lost.
I have found the answer by looking under the Release Pipelines "Initialize Job" log. It appears to substitute the period . with a dash -
The log revealed this...
[SERVICEBUS_PRIMARYRELAY_CONNECTIONNAME] --> [dev-sbrelay]

Enumerate secret variables in Azure Pipelines

I have a build step in Azure Pipelines that takes the variables from Azure Pipelines and uploads them somewhere equally secret. Currently I have about 50 builds, and each build has anywhere between 5-20 variables.
Some are secret and some are not. So for non secret ones I enumerate all the set ones and off i go; but for secret ones I need to add them to the build step manually; further, because I am writing them with the same keys i need to:
Declare variable in the group e.g. MyPrefix.MyVar
Edit the build step to say /specialtool --vars=MyPrefix.MyVar=$(MyPrefix.MyVar) which is rather mundane.
I found that I can get a list of variables using the Azure DevOps api, so i thought i could just modify the next build step as the build is running.
However, if I update the same build definition that is currently running (to dynamically write the command), it is not sent to the agent (rather, it feels like all arguments for tasks are captured when the whole build is triggered). Any thoughts on how i can dynamically enumerate secret vars to feed to my tool?
You can use VSTS Logging Commands to update variable value during the build. This will make the updated variable to be available in next build task.
Write-Host "##vso[task.setvariable variable=testvar;]testvalue"
When you create a Typescript custom task (NodeJS based), you can access all the build variables that are available to the build at that point in time through the getVariable api.
This function returns an array of VariableInfo:
/** Snapshot of a variable at the time when getVariables was called. */
export interface VariableInfo {
name: string;
value: string;
secret: boolean;
}
When you create a PowerShell3 custom task, you can access all the build variables that are available to the build at that point in time through the Get-VstsTaskVariable function.
Which returns a similar object structure as the Node version:
New-Object -TypeName psobject -Property #{
Name = $info.Name
Value = Get-TaskVariable -Name $info.Name
Secret = $info.Secret
}
If you need to support TFS 2015 and the 1.x build agents as well, you can use (now deprecated) PowerShell handler and enumerate the secrets using a custom powershell function I describe here.
Each task SDK (Typescript and Powershell), supports a function to set variables as well. Here is an example of setting the variable value in Typescript:
tl.setVariable(variable, value, isSecret);
And on PowerShell3:
Set-VstsTaskVariable -name $VariableName -value $Value -Secret $IsSecret
And on PowerShell (deprecated):
Write-Host "##vso[task.setvariable variable=$($VariableName);issecret=$($IsSecret)]$Value"
My suspicion is that you'd want to create a single task that reads the variables and invokes the command you mentioned in your original post to then post these variables to the other secret store. It's not recommended to read all the secrets and either store them in non-secret variables or to somehow pass them along to the next task.
So I have been looking at a solution for this too. It appears the only way to do this at the moment is to write a custom task. Within a custom task you can get hold of secret values dynamically.
An example is the 'vsts-replacetokens-task' (https://github.com/qetza/vsts-replacetokens-task/blob/master/task/index.ts)
Internally it uses the vsts task library (vsts-task-lib/task)
(https://github.com/Microsoft/azure-pipelines-task-lib/blob/master/node/task.ts)
This vsts task library exposes methods like GetVariables() and GetVariable() etc. which can provide what you need. Unfortunately bit long winded, but the only way that I can see.

Unable to read VSTS Online release variables

I'm working with VSTS environment variables and stuck with variables of a secret type.
I'm using POSH script (file) to generate a variable (in fact, to obtain the value from Azure Key Vault and the set this value to the variable):
# Add as a script parameter during the release step
-ResourceGroupNameArg "$(ResourceGroupName)" -KeyVaultNameArg "$(KeyVaultName)" -KeyVaultSecretNameArg "$(KeyVaultSecretName)"
# The script itself
Param(
[string]$ResourceGroupNameArg,
[string]$KeyVaultNameArg,
[string]$KeyVaultSecretNameArg
)
<...>
$secret = Get-AzureKeyVaultSecret -VaultName $KeyVaultNameArg -Name $KeyVaultSecretNameArg
$secretValue = $secret.SecretValueText
Write-Host "##vso[task.setvariable variable=SQLAdministratorPassword;issecret=true]$secretValue"
Here I can pass to the script different KeyVault names (according to my needs) - by substituting the $KeyVaultNameArg and $KeyVaultSecretNameArg variables.
For any other variables configured using ##vso[task.setvariable variable= I am able to retrieve them using the construction $env:DatabaseName (for example in another POSH script) or $(DatabaseName) in agent phase step (using Hosted 2017 agent).
However, for the issecret=true variable or even for a manually created variable I'm unable to retrieve its values during the release deployment process.
According to this article,
The values of hidden (secret) variables are stored securely on the
server and cannot be viewed by users after they are saved. During a
deployment, the Release Management service decrypts these values when
referenced by the tasks and passes them to the agent over a secure
HTTPS channel.
So IMO the variables should be accessible for the script (or even agent phase step) despite they are secret.
Refer to these steps to do it:
Click Library tab
Click + Variable group
Specify variable group name
Enable Link secrets for an Azure Key vault as variables and link Azure key vault
Click +Add to add necessary secret(s)
Edit release definition
Choose Variables tab
Select Variable groups
Click Link variable group to link that variable group
Using the related variable directly in release task ($(variable name))