I am trying to access secret variable to pass it to another script.
I expect following code in pipeline to print Value but it prints some text 'xxx' ragardless of the value of a secret variable
echo xxx
Pipeline Snippet
steps:
- bash: echo This script could use $SYSTEM_ACCESSTOKEN
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
If you want to access a secret variable, you could print it to a file. Check the example below:
steps:
- powershell: |
$env:var1 | Out-File C:\Users\xxx\Desktop\Newfolder\debug.txt
displayName: 'PowerShell Script'
env:
var1: $(System.AccessToken)
But System.Accesstoken is a PAT token generated for the service identity “Project Collection Build Service (account)”, it's not needed to verify the value of System.AccessToken. In addition, if you want to print the value of System.AccessToken to a file, you need to check the Allow scripts to access the OAuth token in the agent job:
Azure pipelines will scan the output and mask the secret, you can simply split it up and print it in two parts.
Here is a bash example:
echo "${MY_SECRET:0:10}" # Print the first 10 characters
echo "${MY_SECRET:10:100}" # Print character 11 - 100
You should of course only do it for debugging purposes and not leave it in your pipeline.
Updates:
If I save secret value to a file and publish that file as an artifact secret is visible in cleartext.
After speaking to my colleagues I have realized that all text in logs if it contains a secret value it will be masked.
It will interesting to see if I have 2 variables viz.
OPEN_VAR='something' # No Secret
and
SECRET_VAR='something' # Values same as above but Secret
if I print $OPEN_VAR ; does it mask value because "something" is also a value of "SECRET_VAR"
This is because SYSTEM_ACCESSTOKEN is a secret. If you do the same with variable which is not a secret you will be able to see value.
Related
I am trying to pass a variable "$a" from one step to another step in Azure DevOps. The $hash contains Name,IsReadOnly and Length as NoteProperty. I was successful in passing the variable from one step to next step as $b, but when i try to access NoteProperty of $b variable i am not able to access it
1.My question is it possible pass variable from one step to another step, How do i access the variables NoteProperty ??
2. Does Azure DevOps support passing complex objects from one step to another step ??
For your first question:
is it possible pass variable from one step to another step, How do i access the variables NoteProperty ??
Sure, you can pass variables from a step to the subsequent steps in the same job, and you even can pass the variables to the steps in the subsequent jobs.
To pass a variable value generated in a step to other subsequent steps, you can use the 'SetVariable' command in the step that generates the value.
If you want to share the variable value only to the subsequent steps in the same job, you can just set it as a general job-level variable using the following command.
Bash
echo "##vso[task.setvariable variable=<var_name>;]<var_value>"
or PowerShell
Write-Host "##vso[task.setvariable variable=<var_name>;]<var_value>"
With this way, in the subsequent steps, you can use the expression '$(var_name)' to access this variable.
After executing above command to set up the variable, like as other general pipeline variables, this variable will be automatically mapped as an environment variable. So, you also can use the expression '%VAR_NAME%' (CMD), '$env:VAR_NAME' (PowerShell) or '$VAR_NAME' (Bash) to access the variable.
If you want to share the variable value to the subsequent steps in the same job and the steps in the subsequent jobs, you can set it as an output variable using the following command.
Bash
echo "##vso[task.setvariable variable=<var_name>;isoutput=true]<var_value>"
or PowerShell
Write-Host "##vso[task.setvariable variable=<var_name>;isoutput=true]<var_value>"
With this way, to access this output variable in the subsequent steps of the same job, you can use the expression '$(step_name.var_name)'.
To access this output variable in the steps of the subsequent jobs, the subsequent jobs need to depend on (dependsOn) the job that contains the step to set up the output variable. And you also need to map this output variable as a job-level variable on each subsequent job, and then use this job-level variable to access the output value.
To view more details about variables in Azure Pipelines, you can see "Define variables".
A simple example as reference.
azure-pipelines.yml
jobs:
- job: JobA
displayName: 'Job A'
steps:
- bash: echo "##vso[task.setvariable variable=MyVar;isoutput=true]This is an output variable!"
name: set_output
displayName: 'Set up the output variable'
- bash: echo "set_output.MyVar = $(set_output.MyVar)"
displayName: 'Show the value of output variable'
- job: JobB
displayName: 'Job B'
dependsOn: JobA
variables:
output_from_JobA: $[ dependencies.JobA.outputs['set_output.MyVar'] ]
steps:
- bash: echo "output_from_JobA = $(output_from_JobA)"
displayName: 'Show the value of output variable from JobA'
Result:
For your second question:
Does Azure DevOps support passing complex objects from one step to another step ??
Normally, the value of the variable in Azure Pipelines should be string. To pass the content of an object as a variable in the pipeline, you can try to convert the object (such as JSON node, XML node, etc.) to be a string, then like as above example, pass this string as a variable. In the subsequent steps that need to access this object, after receiving the string variable, you can try to convert the string back to the object. This is the most common method I can think of.
We have certain functional tests that rely on some secrets. Those secrets are obtained from a Azure Key Vault (AKV) and to connect from build agent, I am using environment variables and AzureIdentity.I set those env variables on the build agent machine using powershell. When I use non-secret pipeline variables, then everything works but when I switch to secret pipeline variable for AZURE_CLIENT_SECRET, the authentication starts to fail. I tried the approach of using a script to set the environment variable from secret pipeline variable, but it does not work. I also tried the approach mentioned here but that does not work either. ANy suggestion on how to set an environment variable using secret pipeline variables?
ANy suggestion on how to set an environment variable using secret pipeline variables?
If you set secret variable in below pipeline.
And then use the script's environment or map the variable within the variables block to pass secrets to your pipeline like below script. See: Set secret variables for details.
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Using the mapped env var for this task works and is recommended: $env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(PAT) # the recommended way to map to an env variable
If you use Azure Key vault variable, we create a secret variable(PAT) in below Azure key vault.
So we can link secrets from an Azure key vault in variable group, as below.
Now we can use this variable group in below script. See: Reference secret variables in variable groups for details.
variables:
- group: 'AKVgroup' # variable group
pool:
vmImage: 'ubuntu-latest'
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Using the mapped env var for this task works and is recommended: $env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(PAT) # the recommended way to map to an env variable
The other way is using Azure Key Vault task like below script. See: Use secrets from Azure Key Vault in Azure Pipelines for details.
- task: AzureKeyVault#1
inputs:
azureSubscription: 'ARM'
KeyVaultName: 'edwardkey'
SecretsFilter: '*'
RunAsPreJob: true
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Using the mapped env var for this task works and is recommended: $env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(PAT) # the recommended way to map to an env variable
If you explicitly pass the secret to the script as a parameter then the scrip will have access to it.
If you want to then use that to set an environment variable for use in later scripts you'll can use a different environment variable name and have the script publish that you want it available in subsequent scripts. That sort of defeats the purpose of it being secret but if thats what you want.
If we have an API that accepts a PAT for auth and if we pass System.AccessToken in the header (Authorization) from an AzureDevOps release task when calling this API, what's the suggested way to validate this token on the API side?
System.Accesstoken is secret. If you want to access a secret variable, you could print it to a file. Check the example below:
steps:
- powershell: |
$env:var1 | Out-File C:\Users\xxx\Desktop\Newfolder\debug.txt
displayName: 'PowerShell Script'
env:
var1: $(System.AccessToken)
But System.Accesstoken is a PAT token generated for the service identity “Project Collection Build Service (account)”, it should be valid. Generally, it's not needed to verify the value of System.AccessToken.
If you want to print the value of System.AccessToken to a file, you need to check the Allow scripts to access the OAuth token in the agent job:
I have a Azure Key Vault with different keys (e.g. Key1, Key2, Key3). In some setting, which is dynamically read in one pipeline task, I have value which says which key to use (lets KeyName variable be 'Key2'). How can I read the secret value of the Key2 in the pipeline?
I have tried different combinations and none is working for me.
Test pipeline no. 1 - using the group variable connected to the Key Vault (to make it easier, the KeyName is static here, but in reality, it is set through powershell script during the pipeline):
jobs:
- job: JobA
variables:
- group: KeyVaultGroup #Key vault have secret values "Key1,Key2,Key3..."
- name: KeyName
value: Key2
- name: MyValue
value: $[ variables[variables.KeyName] ]
steps:
- powershell: |
Write-Host "Var1 $($env:VARENV1)"
Write-Host "Var2 $($env:VARENV2)"
env:
VarEnv1: $(MyValue)
VarEnv2: $($(KeyName))
Result is:
Var1
Var2 $(Key2)
MyValue is not working, because the variable is evaluated before the key vault variables are loaded. And when the KeyName is set during the pipeline, it will not work because that too (but this could be solved by using separate job and using output variables to set the KeyName - like in test example no. 2).
Expression $($(KeyName)) is not working, because it will not recursively expand the variable (bug?).
Same problem is when the AzureKeyVault task is used to read the Key Vault values, because it is triggered too late.
Test no. 2 - two separate jobs:
I have used 2 jobs - one to read the key vault and Key name (Job A) and second for the rest (Job B). Problem is, that there is no way how to access the key vault secret values loaded on job A from the job B. I can use only output variables from Job A in the Job B through the dependencies.JobA... but the task AzureKeyVault is not exporting the values as output variables. To do so I will need to use e.g. Powershell task, but in this case, I will need to map the secret values as environment variables into the powershell task, but it means I will loose the dynamic part I need, because it will be statically mapped ( I need to be able to add/remove the values in the key vault without need to change the pipeline). This is no go or I do not know the way how to access the secret variables between jobs without using output variables.
Question:
How to read the secure value from key "Key2" when the Key2 is saved as variable value KeyName and is loaded during the pipeline?
In this case the best way is to use Azure CLI task with azure keyvault command:
- task: AzureCLI#2
inputs:
azureSubscription: 'rg-the-code-manual'
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$secretValue = az keyvault secret show --vault-name tcm-kv --name $(keyName) --query value
echo $secretValue
The content of this script is evaluated at runtime so you can set keyName just before this task and all will be fine. And if you need value of secret as variable you can use logging command to create such.
The easiest way to get rid of double quote will be change output to tsv.
$secretValue = az keyvault secret show --vault-name tcm-kv --name $(keyName) --query value -o tsv
How Can I make Secret variable (SecretVar) defined in azure release pipeline be accessible to my Powershell used to create marketplace task (vsix)?
How Can I make Secret variable (SecretVar) defined in azure release pipeline be accessible to my Powershell used to create marketplace task (vsix)?
You could not access the secret variable directly from the task. This behavior is by designed for protecting secret variables from being exposed in the task.
This documentation states that secret variables are:
Not decrypted into environment variables. So scripts and programs run by your build steps are not given access by default.
Decrypted for access by your build steps. So you can use them in password arguments and also pass them explicitly into a script or a program from your build step (for example as $(password)).
That the reason why you could not use the secret variables in your task.
To resolve this issue, we need to explicitly map secret variables:
variables:
GLOBAL_MYSECRET: $(mySecret)
GLOBAL_MY_MAPPED_ENV_VAR: foo
steps:
- Youtask: |
env:
MY_MAPPED_ENV_VAR: $(mySecret) # right way to map to an env variable
Or if the secret variable can be set as arguments, we could use it:
variables:
VMS_USER: $(vmsUser)
VMS_PASS: $(vmsAdminPass)
pool:
vmImage: 'ubuntu-latest'
steps:
- task: AzureFileCopy#4
inputs:
SourcePath: 'my/path'
azureSubscription: 'my-subscription'
Destination: 'AzureVMs'
storage: 'my-storage'
resourceGroup: 'my-rg'
vmsAdminUserName: $(VMS_USER)
vmsAdminPassword: $(VMS_PASS)
If your task does not support env: or arguments to pass secret variables explicitly into a script, you could not use it in the task.
You could check this thread for and the document for some more details.
Update:
My custom marketplace task uses Powershell (not yaml) and that is
where I would like to access it. How can I do that within powershell?
If you want to access the secret variables in the powershell script instead of the inline/powershell task, you could try to pass the value of secret variable through PowerShell parameters:
Param(
[String]$pass
)
if ($pass) { Write-Host "variable is NOT null" }
if (!$pass) { Write-Host "variable is null" }
Check this thread for some details.
Hope this helps.