In VSTS CI/CD , I am setting some variable's value in a Powershell task in CI.
During CD I want to access that variable's value to do something , lets say echo.
Is this possible? If so, how?
You could write it out to a json/xml file and include that file in your published artifacts of your build defintion. Then read in that file via PowerShell in your release definition.
ConvertTo-Json | Out-File "file.json"
Get-Content "file.json" | ConvertFrom-Json
For VSTS itself, it can not persists variables from build to release.
An workaround is store the variable’s value in Variable Group and link the variable group into your release definition. Detail steps as below:
During build, you can Add a variable group with the name group-$(Build.BuildId), and store the variable you want to transfer in the variable group.
During release, you can get variable groups firstly, and filter the variable under the variable group-$(Build.BuildId). And delete the group group at the end of the release.
Besides, if artifact type is build for your release definition, you can also store the variable value in a file and then publish the file as build artifacts (as Calidus says).
Check out the Azure DevOps extension Variable Tools for Azure DevOps Services.
In the "build pipeline" you can create a JSON file using "save variables". This file needs to be published as artifact or with existing artifact.
In the "release pipeline" you can restore the variables using "load variables" from the JSON file.
Related
I am finding using AZDO Release pipeline variables maddening in Powershell steps.
I am running an Azure PowerShell step to return a primary key value. It is 2 lines…
$primarykey = (Get-AzRelayKey -ResourceGroupName ${env:az-resourcegroupname} -Namespace ${env:az-relaynamespace} -HybridConnection ${env:serviceBus.primaryRelay.ConnectionName} -Name ${env:serviceBus.primaryRelay.KeyName} | Select-Object -ExpandProperty PrimaryKey)
Write-Host "##vso[task.setvariable variable=serviceBus.primaryRelay.Key]$primarykey"
In my pipeline I have a mix of variable names, some I have complete control over (the az- prefixed ones) and others I don’t (the ones starting serviceBus.)
The reason I have no control over the latter is that they are used for a later File Transform step that navigates an appsettings.json file to find/replace values, and its unable to be changed (for example serviceBus.primaryRelay.ConnectionName is a value that is changed in the JSON and the file transform step specifies to navigate the JSON structure, it has to be separated with a period . )
When this script runs it always complains about the -HybridConnection value being empty. This is because the variable has a period in it.
I’ve tried everything I can think of to retrieve that value in the code.
Are they suggesting here that a variable with a period isn’t workable in Powershell in AZDO release pipelines? I’m completely lost.
I have found the answer by looking under the Release Pipelines "Initialize Job" log. It appears to substitute the period . with a dash -
The log revealed this...
[SERVICEBUS_PRIMARYRELAY_CONNECTIONNAME] --> [dev-sbrelay]
I have a complete CI/CD pipeline in Azure DevOps and its working perfectly. now i have a JSON file including the version number for the release.
I need to get this version number as a global variable. How to assign file value to a global variable. I need to use this release no as my build pipeline id, docker tag and release pipeline id.
The way to set a global variable is:
##vso[task.setvariable variable=name;]value
So you can write a PowerShell script that read the release version from the JSON file and set the variable, for example:
$jsonFile = Get-Content path/to/json
$json = $jsonFile | ConvertFrom-Json
$version = $json.release.version
Write-Host "##vso[task.setvariable variable=releaseVersion;]$version"
Now you can use the variable $(version) in your docker tag, etc.
just to add to existing answer, here's how you set build id to the calculater value (because you cannot set it before the build starts, as it is only calculated during the build):
- pwsh: Write-Host "##vso[build.updatebuildnumber]${env:VERSION}"
assuming version is how you called your variable.
I need to use Build.Repository.Uri in a release pipeline. (to pass it to a PowerShell script)
In a buildpipeline:
Write-Host $(Build.Repository.Uri)
> 2019-07-15T08:30:51.8695425Z http://138.202.18.216:8070/Samples/Framework%20A/_git/Framework%20A
In a releasepipeline:
Write-Host $(Build.Repository.Uri)
> The name Build.Repository.Uri was not recognized as the name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if the path is correct (if included), and try again.
Why that inconsistency?
I also try Write-Host $(env:BUILD_REPOSITORY_URI) because of that: How to read directory path of the Artifact in Release pipeline in Azure DevOps? (I also don't understand the logic behind . to _)
Is there a way to get Build.Repository.Uri in a releasepipeline?
EDIT: Solution
"$env:SYSTEM_TASKDEFINITIONSURI$env:BUILD_PROJECTNAME/_git/$env:BUILD_REPOSITORY_NAME" -> http://136.202.18.216:8070/Samples/Framework A/_git/Framework A
If you set system.debug variable to true, you can find all predefined variables inside of the Job Initialize (Auftrag initialisieren) Report after a build.
If your project or repository name contains spaces, make sure that you replace them in your script with %20:
$Uri = $Uri.Replace(" ", "%20")
To access the Build URI in the Release Pipeline you need to use the release variable:
Release.Artifacts.{alias}.BuildURI
{alias} is the the alias of the artifact source you have in the release
If you accessing variables within PowerShell scripts you need to replace any dots with underscores i.e. $env:RELEASE_ARTIFACTS_{alias}_BUILDURI
Source: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch
The variable Build.Repository.Uri is agent-scoped. It can be used as an environment variable in a script and as a parameter in a build task. When you add variable System.Debug with value true in the pipeline, the init job will log all the available environment variables, which includes the REPOSITORY_URI.
You can try with following variables:
Write-Host $env:BUILD_REPOSITORY_URI
Or
Write-Host $env:RELEASE_ARTIFACTS_{alias}_REPOSITORY_URI
Please note that the {alias} is the uppercase of the Artifact source alias.
I have a build step in Azure Pipelines that takes the variables from Azure Pipelines and uploads them somewhere equally secret. Currently I have about 50 builds, and each build has anywhere between 5-20 variables.
Some are secret and some are not. So for non secret ones I enumerate all the set ones and off i go; but for secret ones I need to add them to the build step manually; further, because I am writing them with the same keys i need to:
Declare variable in the group e.g. MyPrefix.MyVar
Edit the build step to say /specialtool --vars=MyPrefix.MyVar=$(MyPrefix.MyVar) which is rather mundane.
I found that I can get a list of variables using the Azure DevOps api, so i thought i could just modify the next build step as the build is running.
However, if I update the same build definition that is currently running (to dynamically write the command), it is not sent to the agent (rather, it feels like all arguments for tasks are captured when the whole build is triggered). Any thoughts on how i can dynamically enumerate secret vars to feed to my tool?
You can use VSTS Logging Commands to update variable value during the build. This will make the updated variable to be available in next build task.
Write-Host "##vso[task.setvariable variable=testvar;]testvalue"
When you create a Typescript custom task (NodeJS based), you can access all the build variables that are available to the build at that point in time through the getVariable api.
This function returns an array of VariableInfo:
/** Snapshot of a variable at the time when getVariables was called. */
export interface VariableInfo {
name: string;
value: string;
secret: boolean;
}
When you create a PowerShell3 custom task, you can access all the build variables that are available to the build at that point in time through the Get-VstsTaskVariable function.
Which returns a similar object structure as the Node version:
New-Object -TypeName psobject -Property #{
Name = $info.Name
Value = Get-TaskVariable -Name $info.Name
Secret = $info.Secret
}
If you need to support TFS 2015 and the 1.x build agents as well, you can use (now deprecated) PowerShell handler and enumerate the secrets using a custom powershell function I describe here.
Each task SDK (Typescript and Powershell), supports a function to set variables as well. Here is an example of setting the variable value in Typescript:
tl.setVariable(variable, value, isSecret);
And on PowerShell3:
Set-VstsTaskVariable -name $VariableName -value $Value -Secret $IsSecret
And on PowerShell (deprecated):
Write-Host "##vso[task.setvariable variable=$($VariableName);issecret=$($IsSecret)]$Value"
My suspicion is that you'd want to create a single task that reads the variables and invokes the command you mentioned in your original post to then post these variables to the other secret store. It's not recommended to read all the secrets and either store them in non-secret variables or to somehow pass them along to the next task.
So I have been looking at a solution for this too. It appears the only way to do this at the moment is to write a custom task. Within a custom task you can get hold of secret values dynamically.
An example is the 'vsts-replacetokens-task' (https://github.com/qetza/vsts-replacetokens-task/blob/master/task/index.ts)
Internally it uses the vsts task library (vsts-task-lib/task)
(https://github.com/Microsoft/azure-pipelines-task-lib/blob/master/node/task.ts)
This vsts task library exposes methods like GetVariables() and GetVariable() etc. which can provide what you need. Unfortunately bit long winded, but the only way that I can see.
I'm working with VSTS environment variables and stuck with variables of a secret type.
I'm using POSH script (file) to generate a variable (in fact, to obtain the value from Azure Key Vault and the set this value to the variable):
# Add as a script parameter during the release step
-ResourceGroupNameArg "$(ResourceGroupName)" -KeyVaultNameArg "$(KeyVaultName)" -KeyVaultSecretNameArg "$(KeyVaultSecretName)"
# The script itself
Param(
[string]$ResourceGroupNameArg,
[string]$KeyVaultNameArg,
[string]$KeyVaultSecretNameArg
)
<...>
$secret = Get-AzureKeyVaultSecret -VaultName $KeyVaultNameArg -Name $KeyVaultSecretNameArg
$secretValue = $secret.SecretValueText
Write-Host "##vso[task.setvariable variable=SQLAdministratorPassword;issecret=true]$secretValue"
Here I can pass to the script different KeyVault names (according to my needs) - by substituting the $KeyVaultNameArg and $KeyVaultSecretNameArg variables.
For any other variables configured using ##vso[task.setvariable variable= I am able to retrieve them using the construction $env:DatabaseName (for example in another POSH script) or $(DatabaseName) in agent phase step (using Hosted 2017 agent).
However, for the issecret=true variable or even for a manually created variable I'm unable to retrieve its values during the release deployment process.
According to this article,
The values of hidden (secret) variables are stored securely on the
server and cannot be viewed by users after they are saved. During a
deployment, the Release Management service decrypts these values when
referenced by the tasks and passes them to the agent over a secure
HTTPS channel.
So IMO the variables should be accessible for the script (or even agent phase step) despite they are secret.
Refer to these steps to do it:
Click Library tab
Click + Variable group
Specify variable group name
Enable Link secrets for an Azure Key vault as variables and link Azure key vault
Click +Add to add necessary secret(s)
Edit release definition
Choose Variables tab
Select Variable groups
Click Link variable group to link that variable group
Using the related variable directly in release task ($(variable name))