Azure DevOps release pipelines let me specify multiple input artifacts but automated triggering only happens for the primary artifact. Is there a way to do a 'wait all' type operation on N input artifacts so that my release pipeline only starts when each artifact has a version available that matches some specified value?
For instance, say I make a commit M in a repo which has multiple different CI jobs A and B which take very different amounts of time. One CI job (A) creates a NuGet package as an artifact in minutes, the other CI job (B) takes hours to create VM images. I want a release pipeline which will use the NuGet package and VM image as inputs. But if I simply use the latest version of A and B as an input artifact in a release pipeline with a CI trigger on the primary artifact (either A or B) then when the release triggers it could be using mismatched versions of artifacts as 10 NuGet packages from 10 different commits are complete before I even get 2 VM images made by the other CI pipeline. I thought maybe build/artifact tags could solve this so that my release waits for artifacts which all have the same tag but I can't use variables in build/artifact tag filters which makes this not really amendable to any sort of automated scheduled/trigger.
Is this some good pattern for this type of workflow?
For this issue , based on your workflow, I am afraid that it is impossible to achieve a fully automated release. Enabling the Continuous deployment trigger will create a new release every time when a new build is available. Since the two artifacts are not generated at the same time, the two matching artifacts cannot be used as the source in automatic release.
To use specific artifacts as source in a release, we usually specify the version in Artifact. But this can't achieve continuous deployment. One way is to merge jobs A and B to generate an artifact, so that you don’t need to worry about the corresponding issues of artifacts, but according to your actual situation, I ’m not sure about the feasibility of this.
do a 'wait all' type operation on N input artifacts so that my release
pipeline only starts when each artifact has a version available that
matches some specified value
Apart from the negative answer, I think what you want is a good idea! So I post a feature request here in DC forum. You could vote that suggestion ticket and share your comment there.The product team would provide the updates if they view it. Thank you for helping us build a better Azure DevOps.
Related
The current scenario is I want a Release Pipeline that has multiple artifacts (one function and several apis). Each one with a cd trigger except for the one to prod which is manual for right now.
In the pipeline is there a way to prevent a particular job from running if deployed pipeline is the same as the original?
For example, a change is made to one of the apis but the rest remain the same I would want the release to get created and deployed but only run the deployment job linked to that particular artifact.
The reason being is that for a deployment to production it would be nice to have all the artifacts I'm looking to deploy in one place and all I would have to do to deploy to prod would be hitting deploy and if the latest artifact is newer then the one deployed it gets deployed but if it is not that deployment would be skipped.
Right now each artifact is in a separate release pipeline and each gets deployed separately, which means we are keeping a separate document with a list of all the release pipelines that needing to be deployed for a specific set of changes.
Any direction would be appreciated, thanks in advance.
I think you can do that with multi-stages and triggers. You can follow these steps:
Create a release pipeline.
Add all your artifacts.
Create multiple stages. According to your requirement, you may need as many stages as your artifacts.
Set triggers for each stage. Click on the lightning icon to the left of the stage. In "Triggers", select "Afer release". Then enable the "Artifact filters". Choose the artifacts that correspond to this stage as the trigger. Here is an example:
Then your release pipeline will be look like this:
When an artifact changes, the corresponding stages will be triggered, and no other stages will run.
It doesn't work. If an artifact is changed it doesnt checked that the artifact is newer
Is it possible to define just one release pipeline for all the packages of one feed ?
A feed contains artifacts of the same kind (i.e. Debian package). Which means the release process usually is identical. Well, at least, in my case it is. So I'd rather not copy/paste release pipelines for each and every package. But rather define a system wide way of deploying artifacts of one kind.
I've tried to set the trigger's package name to wildcard values. The input is accepted but no release is launched.
The release pipeline looks like this:
There are various build pipelines making rpm packages. Those publish the rpm files into a feed (e.g. linux_rpm). The release pipeline I'd like to have is waking up for every artifact in the feed. Instead of waking up for one specific package.
Is it possible to define just one release pipeline for all the
packages of one feed ?
For this issue, the answer is yes. We can add multiple Azure Artifacts type artifacts. In this way, all packages in a feed can be used as release artifacts.
I've configured Release Pipeline with two artifacts linked to it and each artifacts has set to create release whenever new build is available. These two artifact come from two different modules on separate GIT repo's.
In this case, release pipeline will get triggered twice(one per artifact). But I wanted it to get triggered only after all the linked artifact builds are succeeded. Tried finding solution in the docs, but couldn't find it much helpful. Any suggestions please.
I am afraid it is not possible to trigger a release only after all the linked artifact builds are succeeded. Azure devops doesnot have such a feature to configure this yet. A new release will now be triggered when either of the linked artifacts build is succeeded.
You will have to create two different release pipelines, one for each artifact. Or you will have to find a way to put BuildA and BuildB in one artifact.
You can submit a feature request to Microsoft Development team. Hope they will consider implementing this feature in the future sprint. Or you can vote on this existing user voice.
This is not possible to configure it out of the box. It will trigger release just after one of artifact become available. If you want to overcome this, you need to create some custom app to monitor artifacts and trigger release once both artifacts meet some condition.
There is topic in developer community to implement this.
I'm new to Aure DevOps. Trying to create build and release pipelines there's one thing I don't understand:
Commonly, every kind of build finally results in some output, called artifacts.
With Azure DevOps it seems like there is always a final copy or publish task necessary to copy the created artifact from A to B, so the release task may then access the compiled artifacts.
Why aren't these artifacts plain accessible to a release pipeline right from the location where they have been built? Why don't the build tasks automatically set a variable pointing to the right folder, so the release pipeline may access the files right from there?
Or is this already happening and I'm just missing something from the tutorials I watched?
There are so many reasons.
Two easy ones:
There is no guarantee that the agent's working folder still contains the files. Agents are reused from build to build and release to release, and a given build or release will always use the same working folder. The working folder is cleaned up between builds.
Releases may run on different agents. On different machines. In different domains. Or any combination. There's no guarantee that the agent where the build ran is accessible by the agent where the release is running. Publishing the artifact allows a guarantee: As long as the machine the release is running on has the ability to talk to Azure DevOps (which is a requirement for the agent to function in the first place), it can get the artifacts it needs.
Why aren't these artifacts plain accessible to a release pipeline
right from the location where they have been built?
Agree with Daniel.
The main reason for me is because we can't hold the hosted agent all the time. Since MS wants to protect resources efficiently, it is not occupied for a long time.
When we queue a build, MS will assign us a brand new clean agent to execute our task, and after the build is complete, the MS will reclaim the agent assigned to our build and restore the agent to its initial state in preparation for accepting the assignment of the next task.
So, we could not keep hold the hosted agent to use it in next release pipeline. We have to store the artifacts in the cloud/server, then we could download it in the release pipeline. Otherwise, we could not get the artifact we need from an agent that has been restored.
Besides, MS is randomly assigned to the agent, and we cannot guarantee that the same agent will be allocated and built during the release pipeline.
That is the main reason why we need to copy or publish the artifacts.
If you do not want to copy or publish the artifacts, you could setup your own private agent, and do not clean the agent before you execute the release pipeline.
Update:
why is the user, well, bothered to find a place for the artifacts
manually? I would have expected every build pipeline to come with a
personal space to store the latest build artifacts. A space where
Azure DevOps automatically copies the build artifacts to. To me it
looks like things have to be manually copied from A to B and then
later from B to C.
That because not all output is needed, for example, the test project, what we need is test result/Code coverage for the test project, not the output for the solution. In this case, we do not need to copy the output to the artifacts. On the other hand, we need to copy some special files to artfacts, then automatically copy the build artifacts will not meet your requirements.
That is also the reason why we provide the task to copy files to artifacts, so that we could customize our personality needs.
Of course, if you think that manual copying is superfluous, you can use the MSBuild parameter /p:output=$(build.artifactstagingdirectory) to set the output directly to artifacts.
If I need to copy things from A to B in the Build pipeline, then what
should keep me from copying it to C right away? Then a separate
Release pipeline would be, well, rather optional, if not redundant.
If you are in the build pipeline, there is another task Download build artifacts, which could download the build artifacts.
if you are in the release pipeline, you just need select the build artifacts as source, release pipeline will download that artifact automatically:
Check this document for some more details.
Hope this helps.
We are trying to set up an Azure DevOps Pipeline (fka VSTS Release Definition) to accomplish the following flow:
The Production stage (environment) will be triggered if the artifact's build includes a special tag named "Production". This is easily done by adding a artifact filter like so:
The Dev/Test stages (environments) will be triggered if the artifact's build does NOT include the "Production" tag. We are unable to accomplish this since the Exclude filter doesn't allow excluding tags.
The reason we want to skip Dev/Test stages when it's time for us to deploy to Production is because there might be a long time gap between when a build has been validated in Dev/Test and when it's actually Production deployment time. We tag our good build with the "Production" tag and we allow our development to keep going in the master branch. Therefore, when it's time for Production deployment, we don't necessarily want to re-deploy that build to Dev/Test again since Dev/Test might be already several versions ahead.
We know we could also create a Release branch from master to accomplish this goal. However, we'd rather not create Release branches if we could skip Dev/Test deployments using build tags instead.
Please advise. Thanks!
You may be able to achieve the outcome you are looking for but in a different way.
When you create a new release for your pipeline you can change automated triggers to manual triggers.
If you want to skip non production environments so an old version isn't deployed to them, change the automated trigger to manual for those environments.