I have a reasonably complex release pipeline in Azure DevOps that releases a number of Azure apps, a database etc.
Each step is genericised using a library variable for the environment. For example:
But library variables are linked to a release or a selection of stages.
Currently I have to clone the entire pipeline and link a new library variable group in the clone to publish a different environment, but this is heavy on unwanted duplication and maintenance.
How can I run the same release pipeline with different library variables?
If I could do this, it would be possible to have a release for a given branch, for example, but I cannot see a way to do it.
At of this time, it is not supported to select which variable groups to use when you create a release.
If you only have one or several variables, I think you can use pipeline variables instead of variable groups, so that you can update them at release time. Here are the detailed steps:
Go to your pipeline editing page and select "Variables" tab. Click "Add" to add a variable. Then check the option "Settable at release time".
Try re-create your release. You will find the variables defined in #1 and you can edit them before create the release.
If you have many variables, I suggest you try to change the structure of your pipeline to make it more suitable for deployment to multiple environments. As Daniel said, you can use stages for each environment, and then use the variable group in stages scope.
Related
I have 2 build pipelines in my azure devops project, one for building source code and the other one is for
making the setup.
I want the build number generated by the first pipeline that compiles code to be passed to the next pipeline which creates the setup file because i want the setup file to take the same version, so I added a variable group with a variable called sharedBuildCounter.
But when I set sharedBuildCounter the build number in the first pipeline using logging command like this(used inside PowerShell task):
Write-Host "##vso[task.setvariable variable=variable_name;]new_value"
The variable indeed takes the new value and I am able to output the new value using another PowerShell task with one line:
Write-Host $(SharedBuildCounter)
And when I run the next pipeline that builds the setup, I find that sharedBuildCounter is being re-set to the default empty value.
Notice: I found threads that suggests using API rest calls to change variable values, but it don't seem to include a specific pipeline name in case of using pipeline variables(not variable groups).
Variable groups will help to share static values across builds and releases pipeline.
What you need is a way to pass variables from one pipeline to another. I'm afraid to say the is no official way to do this.
As a workaround you could update the value of your variables inside your variable group. There are multiple ways to handle this, Rest API, powershell, 3rd-party extension. Detail ways please refer answers in this question: How to Increase/Update Variable Group value using Azure Devops Build Definition?
If you want to get the value of variable in the pipeline. Since you have used logging command to update that variable.
You need to use Rest API to get that particular build log to fetch related info.
You can use Azure Artifacts to pass information between pipelines. In one pipeline, you write the values to a file and publish the file to an artifact. In the other pipeline, you download the artifact and read the file.
There may be other ways to do it. Azure DevOps allows for free and infinite use of Azure Artifacts in this fashion.
See How to get variable values from pipeline resources in azure pipelines.
I am using the release pipeline classic editor and have a requirement of passing an output variable generated in a task to multiple jobs in the same stage or to outside stages. Currently, this output variable is available only inside the same job and I have to write the same task in multiple jobs and stages and I feel it is a redundancy. Is there any way to implement it?.
In the Classic editor, I am afraid that output variables in a different job is not feasible. Please refer to this document .
As a workaround , you can use variables across jobs and stages via variable groups.
First define the variable in the variable group, then update the variable group through rest api or azure cli, and replace the defined variable with the value of the variable generated by the task.
PUT https://dev.azure.com/{organization}/{project}/_apis/distributedtask/variablegroups/{groupId}?api-version=5.1-preview.1
Here is a case about update variable group with powershell script.
Another workaround : You can share values across all of the stages by using release pipeline variables. The solution is updating the Release Definition for the Release Pipeline variable in the Stage where the variable is set.
Define a variable in the release definition Variable.
Use REST API Definitions - Update to update the value of the
release definition variable in the agent job.
Use the updated value of the release definition variable in the next
agent job.
The details info about using REST API to update the value of the release definition variable, you can follow this ticket.
For detailed steps and guide, please refer to this blog .
We have 100+ services/apps in a repository in Azure Devops. We have defined a single CI/CD YAML multistage pipeline for each (build and deployment). This limits blast radius and allows for auditability of each release of each project. We rely on templates for all the real pipeline work so this is easy to maintain; just a small root azure-pipelines.yml file for each project that includes the needed templates.
Now, we'd like to start using PR validation builds. And, as best as I can tell, we have two options:
Create a separate PR build for for every project and use the UI/API for policies to create 100+ policies
Create a single PR build that has stages for all 100+ projects.
I'm not a fan of the 1st option as now we'll have 200+ builds. The 2nd option is possible, but to avoid a 3 hour PR build, we'd need a way to only run needed stages (aka project builds).
Is there a 3rd option I'm missing? If the 2nd option is our best bet, how do we turn off stages for projects not changed in that PR (i.e. what condition would we use)?
(FYI, our policy is to change only one project per PR, but there are, on occasion exceptions to that.)
For personal suggestion, I also recommend the second method. Though the build script would be very large in one configure file, but much better than have hundreds build configuration files.
But the difficulty is these 100+ apps are all in one repository. This means all the normal method will not suitable for you, include using Build.Repository.Name value as the stage condition. Also, there's no more details which describing the source file path stored in the commit.
So, I suggest you and your team developers input the project name info into your commit message. Then, in the build pipeline you could use the variable Build.SourceVersionMessage to get its comment message. Since this is a environment variable which only work in step level(Not work for stage level and the job level), it needs you add one task in the first step and use the condition for it.
The logic of it is add one step as the first one in every stages. This step is only used to conditional judgment. If the Build.SourceVersionMessage matches the prefix or any key contents words, the jobs will be early-exit.
If use the condition like this:
condition: startsWith(variables['Build.SourceVersionMessage'], '[maven-plugin]')
It needs your commit message must follow a strict content writing format, starting with the specified project name.
Another condition can for you consider is:
condition: in(variables['Build.SourceVersionMessage'], 'maven-plugin')
This does not need the strict content writing format, but also need input the project name in the commit message. Thus it could be evaluated in the job condition with the above script.
Hope it could give you some help.
I have more than 35 projects in an organization in Azure DevOps. These are the solutions of a Single product. Right now I am creating build and release pipelines for each project one by one.
If I create a pipeline for a single project and release the build, I want devops to create pipeline & releases builds for all the other projects(in the same organization) together at once.
For eg, There are projects A,B,C & D. If I create a release pipeline for Project A, will Devops automatically Release build for other projects "B,C & D" in the same organization at the same time.
We need to avoid creating pipelines for each projects one by one. Is this possible and is there any scripting or configuration to achieve this?
Thanks in Advance.
The process of setting up multiple release definitions with the exact same tasks can be very time consuming and difficult to manage -- fortunately, our team solved this problem using Task Groups!
Task Groups allow you to bundle multiple steps into a single "group", parameterize them, and then invoke them like a single step. You can also edit the Task Group at any time which will cascade to all places where it's used.
The other consideration is that you can bundle your custom variables into Variable Groups and reuse them between Release Definitions. Having your configuration reusable also has the benefit of only having to edit a single item.
I'd recommend:
Create a single release definition as a golden reference example.
Add your build steps for a single stage/environment
Once you're happy with this release definition, multi-select the steps you want to bundle into a group, open the context menu and select the option to convert to a Task Group. This will remove the steps from your release and move them to the Task Group.
Customize the Task Group with the appropriate parameters and then save it.
Modify your release definition to use the Task group parameters.
Add the additional stages to your environment, and add your custom Task Group to each parameterizing with configuration specific to the environment.
Once you're happy with the Release Definition -- use the Clone feature to create 34 more instances.
The same approach can be applied to builds.
I am configuring a release pipeline in Azure DevOps and I want the variables that get generated along the tasks to persist across re-execution of the same release. I wanted to know if that is possible.
The main goal is to create a pipeline that i can redeploy in case of a failure, if for example I have a release pipeline with 30 tasks, I would want handle skipping the tasks that got completed, but once I reach the relevant task, I need the persisted variable values.
I have looked online and I see it isn't possible to persist variables across phases, but does that also mean it cannot be persisted in the same release pipeline if I redeploy it?
From searching stack exchange and google I got to the following GitHub issue on the subject, I just wasn't sure if it also affects my own situation in the same way.
https://github.com/Microsoft/azure-pipelines-tasks/issues/4743
You have that by default unless I interpret you wrong. When redeploying the same release pipeline variables values you define (in the pipeline) do not change.
Calculated values are not persisted