How Can I make Secret variable (SecretVar) defined in azure release pipeline be accessible to my Powershell used to create marketplace task (vsix)?
How Can I make Secret variable (SecretVar) defined in azure release pipeline be accessible to my Powershell used to create marketplace task (vsix)?
You could not access the secret variable directly from the task. This behavior is by designed for protecting secret variables from being exposed in the task.
This documentation states that secret variables are:
Not decrypted into environment variables. So scripts and programs run by your build steps are not given access by default.
Decrypted for access by your build steps. So you can use them in password arguments and also pass them explicitly into a script or a program from your build step (for example as $(password)).
That the reason why you could not use the secret variables in your task.
To resolve this issue, we need to explicitly map secret variables:
variables:
GLOBAL_MYSECRET: $(mySecret)
GLOBAL_MY_MAPPED_ENV_VAR: foo
steps:
- Youtask: |
env:
MY_MAPPED_ENV_VAR: $(mySecret) # right way to map to an env variable
Or if the secret variable can be set as arguments, we could use it:
variables:
VMS_USER: $(vmsUser)
VMS_PASS: $(vmsAdminPass)
pool:
vmImage: 'ubuntu-latest'
steps:
- task: AzureFileCopy#4
inputs:
SourcePath: 'my/path'
azureSubscription: 'my-subscription'
Destination: 'AzureVMs'
storage: 'my-storage'
resourceGroup: 'my-rg'
vmsAdminUserName: $(VMS_USER)
vmsAdminPassword: $(VMS_PASS)
If your task does not support env: or arguments to pass secret variables explicitly into a script, you could not use it in the task.
You could check this thread for and the document for some more details.
Update:
My custom marketplace task uses Powershell (not yaml) and that is
where I would like to access it. How can I do that within powershell?
If you want to access the secret variables in the powershell script instead of the inline/powershell task, you could try to pass the value of secret variable through PowerShell parameters:
Param(
[String]$pass
)
if ($pass) { Write-Host "variable is NOT null" }
if (!$pass) { Write-Host "variable is null" }
Check this thread for some details.
Hope this helps.
Related
As part of build I am using 'envsubst' command to replace all secrets from environment variables to my application configuration file. We are using Azure DevOps pipeline for our build process and now start using Azure KeyVault to store all these Secrets. Current Issue is that I am not able to make these secrets as environment variable in MS based agent runner. I tried to refer multiple documents but nothing helps.
Did anyone able to set the Azure KeyVault secrets as environment variable on the build agent runner. Any clue/guidance will be of help
Tried using setvariable task but it helps to read the secret and use it within job but not help insetting that variable as environment variable
task: Bash#3
inputs:
targetType: 'inline'
script: |
# Write your commands here
echo "##vso[task.setvariable variable=MySecret;issecret=true]$(MY-SECRET)"
Export and set command inside shell task is not allowing to setup environment variable
Also tried env setting, but that too didn't help
env:
MYSECRET: $(MY-SECRET)
Anyone who have implemented the same, do let me know
I am trying to create a dynamic variable on Azure Devops pipeline and I cannot use it on further tasks although the task with the variable creation is successfully completed.
Write-Host "##vso[task.setvariable variable=mytitle;isOutput=true]$content
How can I use the variable that is created dynamically with setvariable
Please note that the first task can set a variable, and following tasks are able to use the variable using macro syntax $(myVar). We are not able to use it in the first task. The variable will only be available to tasks in the same job by default, and the variable is exposed to the following tasks as an environment variable.
For example:
jobs:
- job: A
steps:
- powershell: |
Write-Host "##vso[task.setvariable variable=mytitle]$content"
name: setVariable
- powershell: |
Write-Host "You can use macro syntax for variables: $(mytitle)"
name: PrintVariable
If you add the parameter isoutput, the syntax to call your variable changes. See Set an output variable for use in the same job and Set an output variable for use in future jobs for details.
More information please refer to Logging commands.
We have certain functional tests that rely on some secrets. Those secrets are obtained from a Azure Key Vault (AKV) and to connect from build agent, I am using environment variables and AzureIdentity.I set those env variables on the build agent machine using powershell. When I use non-secret pipeline variables, then everything works but when I switch to secret pipeline variable for AZURE_CLIENT_SECRET, the authentication starts to fail. I tried the approach of using a script to set the environment variable from secret pipeline variable, but it does not work. I also tried the approach mentioned here but that does not work either. ANy suggestion on how to set an environment variable using secret pipeline variables?
ANy suggestion on how to set an environment variable using secret pipeline variables?
If you set secret variable in below pipeline.
And then use the script's environment or map the variable within the variables block to pass secrets to your pipeline like below script. See: Set secret variables for details.
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Using the mapped env var for this task works and is recommended: $env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(PAT) # the recommended way to map to an env variable
If you use Azure Key vault variable, we create a secret variable(PAT) in below Azure key vault.
So we can link secrets from an Azure key vault in variable group, as below.
Now we can use this variable group in below script. See: Reference secret variables in variable groups for details.
variables:
- group: 'AKVgroup' # variable group
pool:
vmImage: 'ubuntu-latest'
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Using the mapped env var for this task works and is recommended: $env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(PAT) # the recommended way to map to an env variable
The other way is using Azure Key Vault task like below script. See: Use secrets from Azure Key Vault in Azure Pipelines for details.
- task: AzureKeyVault#1
inputs:
azureSubscription: 'ARM'
KeyVaultName: 'edwardkey'
SecretsFilter: '*'
RunAsPreJob: true
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Using the mapped env var for this task works and is recommended: $env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(PAT) # the recommended way to map to an env variable
If you explicitly pass the secret to the script as a parameter then the scrip will have access to it.
If you want to then use that to set an environment variable for use in later scripts you'll can use a different environment variable name and have the script publish that you want it available in subsequent scripts. That sort of defeats the purpose of it being secret but if thats what you want.
Is there a way to pass the Azure DevOps pipeline secrets without passing them as task env variables
As per the documentation, we can only pass the secrets like below
- powershell: |
Write-Host "recommended: $env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(mySecret) # the recommended way to map to an env variable
This works okay when you have 1 or 2 variables, but managing 20 or more than that becomes so tedious.
Any comments or suggestions would be greatly appreciated
Update:
Is there a way to pass the Azure DevOps pipeline secrets without passing them as task env variables
I'm afraid there is no method to use secret variables directly without mapping them to the task environment.
From the document you mentioned：
Don't set secret variables in your YAML file. Operating systems often
log commands for the processes that they run, and you wouldn't want
the log to include a secret that you passed in as an input. Use the
script's environment or map the variable within the variables block to
pass secrets to your pipeline.
We don't recommend passing the secret variables directly into pipeline. So we could use the env variable to map the secret variable.
Generally, we only place a small number of secret variables in a single task.
Therefore, it is valuable to use environment variables to map secret variables.
I have a Azure Key Vault with different keys (e.g. Key1, Key2, Key3). In some setting, which is dynamically read in one pipeline task, I have value which says which key to use (lets KeyName variable be 'Key2'). How can I read the secret value of the Key2 in the pipeline?
I have tried different combinations and none is working for me.
Test pipeline no. 1 - using the group variable connected to the Key Vault (to make it easier, the KeyName is static here, but in reality, it is set through powershell script during the pipeline):
jobs:
- job: JobA
variables:
- group: KeyVaultGroup #Key vault have secret values "Key1,Key2,Key3..."
- name: KeyName
value: Key2
- name: MyValue
value: $[ variables[variables.KeyName] ]
steps:
- powershell: |
Write-Host "Var1 $($env:VARENV1)"
Write-Host "Var2 $($env:VARENV2)"
env:
VarEnv1: $(MyValue)
VarEnv2: $($(KeyName))
Result is:
Var1
Var2 $(Key2)
MyValue is not working, because the variable is evaluated before the key vault variables are loaded. And when the KeyName is set during the pipeline, it will not work because that too (but this could be solved by using separate job and using output variables to set the KeyName - like in test example no. 2).
Expression $($(KeyName)) is not working, because it will not recursively expand the variable (bug?).
Same problem is when the AzureKeyVault task is used to read the Key Vault values, because it is triggered too late.
Test no. 2 - two separate jobs:
I have used 2 jobs - one to read the key vault and Key name (Job A) and second for the rest (Job B). Problem is, that there is no way how to access the key vault secret values loaded on job A from the job B. I can use only output variables from Job A in the Job B through the dependencies.JobA... but the task AzureKeyVault is not exporting the values as output variables. To do so I will need to use e.g. Powershell task, but in this case, I will need to map the secret values as environment variables into the powershell task, but it means I will loose the dynamic part I need, because it will be statically mapped ( I need to be able to add/remove the values in the key vault without need to change the pipeline). This is no go or I do not know the way how to access the secret variables between jobs without using output variables.
Question:
How to read the secure value from key "Key2" when the Key2 is saved as variable value KeyName and is loaded during the pipeline?
In this case the best way is to use Azure CLI task with azure keyvault command:
- task: AzureCLI#2
inputs:
azureSubscription: 'rg-the-code-manual'
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$secretValue = az keyvault secret show --vault-name tcm-kv --name $(keyName) --query value
echo $secretValue
The content of this script is evaluated at runtime so you can set keyName just before this task and all will be fine. And if you need value of secret as variable you can use logging command to create such.
The easiest way to get rid of double quote will be change output to tsv.
$secretValue = az keyvault secret show --vault-name tcm-kv --name $(keyName) --query value -o tsv