I am finding using AZDO Release pipeline variables maddening in Powershell steps.
I am running an Azure PowerShell step to return a primary key value. It is 2 lines…
$primarykey = (Get-AzRelayKey -ResourceGroupName ${env:az-resourcegroupname} -Namespace ${env:az-relaynamespace} -HybridConnection ${env:serviceBus.primaryRelay.ConnectionName} -Name ${env:serviceBus.primaryRelay.KeyName} | Select-Object -ExpandProperty PrimaryKey)
Write-Host "##vso[task.setvariable variable=serviceBus.primaryRelay.Key]$primarykey"
In my pipeline I have a mix of variable names, some I have complete control over (the az- prefixed ones) and others I don’t (the ones starting serviceBus.)
The reason I have no control over the latter is that they are used for a later File Transform step that navigates an appsettings.json file to find/replace values, and its unable to be changed (for example serviceBus.primaryRelay.ConnectionName is a value that is changed in the JSON and the file transform step specifies to navigate the JSON structure, it has to be separated with a period . )
When this script runs it always complains about the -HybridConnection value being empty. This is because the variable has a period in it.
I’ve tried everything I can think of to retrieve that value in the code.
Are they suggesting here that a variable with a period isn’t workable in Powershell in AZDO release pipelines? I’m completely lost.
I have found the answer by looking under the Release Pipelines "Initialize Job" log. It appears to substitute the period . with a dash -
The log revealed this...
[SERVICEBUS_PRIMARYRELAY_CONNECTIONNAME] --> [dev-sbrelay]
Related
I have a pipeline variable called TestVariable.
I can easily access this variable from a PS script like so:
write-host $(TestVariable)
But if the name of that variable was dynamic, is there any way I can access the variable value from PS?
For example, the name of the variable would go into a string variable. I've tried these combinations as experiments...they just return the variable name, not the value (not surprisingly):
$varname="TestVariable"
write-host $($varname)
write-host $("$varname")
write-host $"($varname)"
write-host $("($varname)")
I think the answer is no, but I want to be sure. Any help much appreciated!
Edit - note
Both answers answer the question but don't solve my problem. After trying the solutions I realized I missed an additional complication which the answers don't help with unfortunately. Am noting here in case someone tries to do something similar.
The extra complication is, the value of the variable is set during the release (I'm trying to access ARM template output variables).
I thought I may be able to hit the API and get the 'live' variable value but unfortunately the release data does not exist (from the API) until the release completes.
So when I call this during a release:
https://vsrm.dev.azure.com/{company}/{project}/_apis/release/releases/$($releaseId)?api-version=5.0
I get "Release with ID 38 does not exist".
Late to the party, but figure I'd share.
As mentioned in the Defined Variables doc, pipeline variables are accessible through
the environment variables. While $(varname) gets processed before the task starts, $env:varname can be invoked mid-run. So you can cheat by using:
Write-Host ('$env:'+"$(varname)" | Invoke-Expression)
The task will resolve $(varname) into its value before the task begins. So the script reads as
Write-Host ("$env:TestVariable" | Invoke-Expression)
And it'll spit out the same as calling $(TestVariable).
Though you do need to respect the rules, such as " " and "." -> "_".
Dynamic variable name in VSTS (Azure DevOps) pipeline
Agree with Krzysztof Madej. There is no out of box way to achieve this.
That because the nested variables (like $($varname) are not yet supported in the build pipelines.
To resolve this issue, you could use the Definitions - Get to get the value of Dynamic variable:
GET https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=5.1
Below is my test powershell script:
$varname="TestVariable"
$url = "https://dev.azure.com/YourOrganizationName/YourtProjectName/_apis/build/definitions/<definitionsId>?api-version=5.0"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
$VFDV= $pipeline.variables.$varname.value
Write-Host This is Value For Dynamic Variable: $VFDV
The output:
Hope this helps.
This is not possible directly in YAML but you can use for instance az cli. With this you can set programatically a variable name a get value of it
$variableName = "some"
az pipelines variable list --org "https://dev.azure.com/thecodemanual" --project "DevOps Manual" --pipeline-name "DevOps Manual-CI" --query ($variableName + '.value')
$variableName = "test"
az pipelines variable list --org "https://dev.azure.com/thecodemanual" --project "DevOps Manual" --pipeline-name "DevOps Manual-CI" --query ($variableName + '.value')
Now you can use this code in a powershell task to fetch value of variable.
Here you have info how to install extension.
I know its too late but I can tell you a perfect solution. as we know all the pipeline variables are available as environment variables so we can access the values as below
var=$(TestVariable)
upper= ${var^^} //convert the variable to uppercase as env variables are upper
echo ${!upper}
Please note the above solution is tested and works in bash only. I haven't written for PS
I need to use Build.Repository.Uri in a release pipeline. (to pass it to a PowerShell script)
In a buildpipeline:
Write-Host $(Build.Repository.Uri)
> 2019-07-15T08:30:51.8695425Z http://138.202.18.216:8070/Samples/Framework%20A/_git/Framework%20A
In a releasepipeline:
Write-Host $(Build.Repository.Uri)
> The name Build.Repository.Uri was not recognized as the name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if the path is correct (if included), and try again.
Why that inconsistency?
I also try Write-Host $(env:BUILD_REPOSITORY_URI) because of that: How to read directory path of the Artifact in Release pipeline in Azure DevOps? (I also don't understand the logic behind . to _)
Is there a way to get Build.Repository.Uri in a releasepipeline?
EDIT: Solution
"$env:SYSTEM_TASKDEFINITIONSURI$env:BUILD_PROJECTNAME/_git/$env:BUILD_REPOSITORY_NAME" -> http://136.202.18.216:8070/Samples/Framework A/_git/Framework A
If you set system.debug variable to true, you can find all predefined variables inside of the Job Initialize (Auftrag initialisieren) Report after a build.
If your project or repository name contains spaces, make sure that you replace them in your script with %20:
$Uri = $Uri.Replace(" ", "%20")
To access the Build URI in the Release Pipeline you need to use the release variable:
Release.Artifacts.{alias}.BuildURI
{alias} is the the alias of the artifact source you have in the release
If you accessing variables within PowerShell scripts you need to replace any dots with underscores i.e. $env:RELEASE_ARTIFACTS_{alias}_BUILDURI
Source: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch
The variable Build.Repository.Uri is agent-scoped. It can be used as an environment variable in a script and as a parameter in a build task. When you add variable System.Debug with value true in the pipeline, the init job will log all the available environment variables, which includes the REPOSITORY_URI.
You can try with following variables:
Write-Host $env:BUILD_REPOSITORY_URI
Or
Write-Host $env:RELEASE_ARTIFACTS_{alias}_REPOSITORY_URI
Please note that the {alias} is the uppercase of the Artifact source alias.
I have a build step in Azure Pipelines that takes the variables from Azure Pipelines and uploads them somewhere equally secret. Currently I have about 50 builds, and each build has anywhere between 5-20 variables.
Some are secret and some are not. So for non secret ones I enumerate all the set ones and off i go; but for secret ones I need to add them to the build step manually; further, because I am writing them with the same keys i need to:
Declare variable in the group e.g. MyPrefix.MyVar
Edit the build step to say /specialtool --vars=MyPrefix.MyVar=$(MyPrefix.MyVar) which is rather mundane.
I found that I can get a list of variables using the Azure DevOps api, so i thought i could just modify the next build step as the build is running.
However, if I update the same build definition that is currently running (to dynamically write the command), it is not sent to the agent (rather, it feels like all arguments for tasks are captured when the whole build is triggered). Any thoughts on how i can dynamically enumerate secret vars to feed to my tool?
You can use VSTS Logging Commands to update variable value during the build. This will make the updated variable to be available in next build task.
Write-Host "##vso[task.setvariable variable=testvar;]testvalue"
When you create a Typescript custom task (NodeJS based), you can access all the build variables that are available to the build at that point in time through the getVariable api.
This function returns an array of VariableInfo:
/** Snapshot of a variable at the time when getVariables was called. */
export interface VariableInfo {
name: string;
value: string;
secret: boolean;
}
When you create a PowerShell3 custom task, you can access all the build variables that are available to the build at that point in time through the Get-VstsTaskVariable function.
Which returns a similar object structure as the Node version:
New-Object -TypeName psobject -Property #{
Name = $info.Name
Value = Get-TaskVariable -Name $info.Name
Secret = $info.Secret
}
If you need to support TFS 2015 and the 1.x build agents as well, you can use (now deprecated) PowerShell handler and enumerate the secrets using a custom powershell function I describe here.
Each task SDK (Typescript and Powershell), supports a function to set variables as well. Here is an example of setting the variable value in Typescript:
tl.setVariable(variable, value, isSecret);
And on PowerShell3:
Set-VstsTaskVariable -name $VariableName -value $Value -Secret $IsSecret
And on PowerShell (deprecated):
Write-Host "##vso[task.setvariable variable=$($VariableName);issecret=$($IsSecret)]$Value"
My suspicion is that you'd want to create a single task that reads the variables and invokes the command you mentioned in your original post to then post these variables to the other secret store. It's not recommended to read all the secrets and either store them in non-secret variables or to somehow pass them along to the next task.
So I have been looking at a solution for this too. It appears the only way to do this at the moment is to write a custom task. Within a custom task you can get hold of secret values dynamically.
An example is the 'vsts-replacetokens-task' (https://github.com/qetza/vsts-replacetokens-task/blob/master/task/index.ts)
Internally it uses the vsts task library (vsts-task-lib/task)
(https://github.com/Microsoft/azure-pipelines-task-lib/blob/master/node/task.ts)
This vsts task library exposes methods like GetVariables() and GetVariable() etc. which can provide what you need. Unfortunately bit long winded, but the only way that I can see.
In VSTS CI/CD , I am setting some variable's value in a Powershell task in CI.
During CD I want to access that variable's value to do something , lets say echo.
Is this possible? If so, how?
You could write it out to a json/xml file and include that file in your published artifacts of your build defintion. Then read in that file via PowerShell in your release definition.
ConvertTo-Json | Out-File "file.json"
Get-Content "file.json" | ConvertFrom-Json
For VSTS itself, it can not persists variables from build to release.
An workaround is store the variable’s value in Variable Group and link the variable group into your release definition. Detail steps as below:
During build, you can Add a variable group with the name group-$(Build.BuildId), and store the variable you want to transfer in the variable group.
During release, you can get variable groups firstly, and filter the variable under the variable group-$(Build.BuildId). And delete the group group at the end of the release.
Besides, if artifact type is build for your release definition, you can also store the variable value in a file and then publish the file as build artifacts (as Calidus says).
Check out the Azure DevOps extension Variable Tools for Azure DevOps Services.
In the "build pipeline" you can create a JSON file using "save variables". This file needs to be published as artifact or with existing artifact.
In the "release pipeline" you can restore the variables using "load variables" from the JSON file.
How can I get a current date-time and pass it as a variable to some Deployment task?
You can define a variable with any value, and then modify the variable as current date. Detail steps as below:
Define a variable in release
Assume the variable name is time, and we set the value as none. If you need to use the variable for a environment, you can define it in environment variables. Else you should define it in variables Tab.
Add a power shell task at the begin of deploy tasks:
Type: Inline Script.
Inline script:
$date=$(Get-Date -Format g);
Write-Host "##vso[task.setvariable variable=time]$date"
Note:
I use the date format as MM/DD/YYYY HH:MM AM/PM here. You can use other date formats.
For the subsequent deploy task, if you want to use current date time, you can direct use $(time).
Update
Documentation for Defining Variables: Set Variables Using Expressions has a nugget of gold for the answer to this question in the example for creating a counter value that is reset daily.
a: $[counter(format('{0:yyyyMMdd}', pipeline.startTime), 100)]
The pipeline.startTime variable used here is subtle and not mentioned anywhere in the Pipeline Predefined Variables documentation, even when being careful to land on the correct documentation based on the pipeline method being used. As is suggested HERE and in some of the answers on this thread, certain variables may have different values or not exist at all depending on where you are while trying to access them.
There is now a variable specific to a release stage named "Release.Deployment.StartTime" or if you use it in powershell the environment variable is "Release_Deployment_StartTime".
It's in UTC and the format is "yyyy-MM-dd HH:mm:ssZ"
ex: "2018-11-09 21:23:27Z"
NOTE: This variable is set at the time the deployment stage is started, so if you have pre-deployment approvals the time will be set before any approvals are completed. From my testing if you have multiple stages that execute at the same time it will be the same between them, even if one stage waits for the other due to limited agent availability.
I'm using Azure DevOps online, unsure if local TFS installations will have this.
For those who use Linux on tfs:
Define variable
Make sure it has "Settable at queue time set"
Create a script in root of your repository
set-build.date.sh:
#!/usr/bin/env bash
DATE=$(date '+%d/%m/%Y %H:%M:%S')
echo "##vso[task.setvariable variable=BUILD_DATE;]$DATE"
Other options are listed here.
Add shell script right after get sources
Type bash to find this task.
Done, you can use BUILD_DATE variable in later tasks :)
An easier way is
$(Date:MMddyy)
Some options are only available in the Build Definition options section. The date formatting is one of them. However, if you were to go into the options section, set the build number format as $(Date:yyyyMMdd-HHmmss), you could then use the $(Build.BuildNumber) variable in your tasks.
More info here - https://learn.microsoft.com/en-us/vsts/build-release/concepts/definitions/build/variables?tabs=batch
Based on answer of Marina Liu (here) for quick copy and paste.
Define variable on top:
variables:
buildTimeStamp: # will be set by script
# ...
Add this as first task (and change format as needed):
- task: PowerShell#2
displayName: set variable buildTimeStamp
inputs:
targetType: 'inline'
script: |
$date=$(Get-Date -Format yyyy-MM-dd_HH.mm);
Write-Host "##vso[task.setvariable variable=buildTimeStamp]$date"
Now you can use variable $(buildTimeStamp) in your code below.