Is it possible to define just one release pipeline for all the packages of one feed ?
A feed contains artifacts of the same kind (i.e. Debian package). Which means the release process usually is identical. Well, at least, in my case it is. So I'd rather not copy/paste release pipelines for each and every package. But rather define a system wide way of deploying artifacts of one kind.
I've tried to set the trigger's package name to wildcard values. The input is accepted but no release is launched.
The release pipeline looks like this:
There are various build pipelines making rpm packages. Those publish the rpm files into a feed (e.g. linux_rpm). The release pipeline I'd like to have is waking up for every artifact in the feed. Instead of waking up for one specific package.
Is it possible to define just one release pipeline for all the
packages of one feed ?
For this issue, the answer is yes. We can add multiple Azure Artifacts type artifacts. In this way, all packages in a feed can be used as release artifacts.
Related
I am trying to accomplish the following:
When I kick off a release of a build, I want to include the previous release with the latest one. What I am trying to do is run a script inside the release that compares the two files and outputs a combined file with additions and deletions highlighted. I have a script that will accomplish that, but im not entirely sure how to get yesterdays artifact with todays artifact do complete the release.
I was thinking about tagging but maybe someone has a creative idea.
Using tags is a way to distinguish the dates of artifacts.
The artifact itself also has a parameter to distinguish between versions.
In Release Pipeline, you could add two artifacts from the same Build Pipeline.
For example:
The version of artifacts is distinguished by build number. You could select the Specific version and choose the target version for the two artifacts.
Then you could download the old and new artifacts in one release pipeline.
Note: You need to change the Source Alias for each artifacts. They need to be different.
Azure DevOps release pipelines let me specify multiple input artifacts but automated triggering only happens for the primary artifact. Is there a way to do a 'wait all' type operation on N input artifacts so that my release pipeline only starts when each artifact has a version available that matches some specified value?
For instance, say I make a commit M in a repo which has multiple different CI jobs A and B which take very different amounts of time. One CI job (A) creates a NuGet package as an artifact in minutes, the other CI job (B) takes hours to create VM images. I want a release pipeline which will use the NuGet package and VM image as inputs. But if I simply use the latest version of A and B as an input artifact in a release pipeline with a CI trigger on the primary artifact (either A or B) then when the release triggers it could be using mismatched versions of artifacts as 10 NuGet packages from 10 different commits are complete before I even get 2 VM images made by the other CI pipeline. I thought maybe build/artifact tags could solve this so that my release waits for artifacts which all have the same tag but I can't use variables in build/artifact tag filters which makes this not really amendable to any sort of automated scheduled/trigger.
Is this some good pattern for this type of workflow?
For this issue , based on your workflow, I am afraid that it is impossible to achieve a fully automated release. Enabling the Continuous deployment trigger will create a new release every time when a new build is available. Since the two artifacts are not generated at the same time, the two matching artifacts cannot be used as the source in automatic release.
To use specific artifacts as source in a release, we usually specify the version in Artifact. But this can't achieve continuous deployment. One way is to merge jobs A and B to generate an artifact, so that you don’t need to worry about the corresponding issues of artifacts, but according to your actual situation, I ’m not sure about the feasibility of this.
do a 'wait all' type operation on N input artifacts so that my release
pipeline only starts when each artifact has a version available that
matches some specified value
Apart from the negative answer, I think what you want is a good idea! So I post a feature request here in DC forum. You could vote that suggestion ticket and share your comment there.The product team would provide the updates if they view it. Thank you for helping us build a better Azure DevOps.
I need to deploy many similar apps to the same environments with Azure DevOps.
What are some ways to share and maintain the release process for these apps....
to avoid updating every app's Release process when it changes?
Background
I have dozens of similar apps with identical release processes.
Each app (in its own repo) will have its own Azure Pipeline.
I'm fine with a custom solution to this problem.
Options
Make a Release for every Pipeline -- not a fan!
CON: Azure seems to want a 1-to-1 relationship between Pipelines and Releases.
CON: I want to avoid many Releases at all costs, since changes would be nearly unmaintainable.
Use custom stage templates -- doesn't work for what I need
CON: Releases do not share custom stage templates.
Custom templates that you create are scoped to the project that you created them in. [Azure Documentation]
CON: Custom stage templates cannot be updated (as of this post).
To update an stage template, delete the existing template in a release pipeline and then save the stage as a template with the same name. [Azure Documentation]
Put the release process in the Pipeline .yml file -- seems possible, but....
PRO: The process would be stored in a shared "common" repo.
PRO: Each app will have a minimal Pipeline .yml file, setting some parameters before running the shared process.
CON: How would I track and manually deploy releases without the Release portal UI? Am I missing something?
Trigger the same Release with artifacts from different apps -- not sure this will work....
PRO: One release process, easy to track and deploy as needed via the Release portal UI.
CON: How would I track releases for different apps? Would I name Releases with the app name?
CON: I don't see how to set it up, since a Release is tied to a primary Pipeline.
At the time of linking an artifact source to a release pipeline.... [Azure Documentation]
CON: Though multiple artifact sources can be used, Releases need a default artifact Pipeline or source repo. Would I need to setup the artifact source programmatically?
When you link multiple artifact sources to a release pipeline, one of them is designated as the primary artifact source. The primary artifact source is used to set a number of pre-defined variables. It can also be used in naming releases. [Azure Documentation]
Use multiple artifact sources and artifact variables -- not sure this will work....
When there are multiple artifact sources linked to a release pipeline, you can access information about each of these. [Azure Documentation]
Bottom Line
I was able to solve many of these problems in Octopus Deploy. However, I'm having significant trouble seeing how to move my DevOps process into Azure DevOps. How would you handle this situation?
There is no perfect solution to meet your requirements.
Generally deploy multiple apps to multiple environments, we recommend that Make a Release for every Pipeline. But just as you mentioned it has 1-to-1 relationship between build pipelines and Releases and it has many releases.
We can deploy multiple apps to multiple environments through ONE release with customized Template as you mentioned, we can also use the Task Groups to combine a set of tasks together, then reuse them in different stages. But we still need to update the definitions accordingly (change the Task Group version) once the specific task group is updated.
Another way is Clone Stage, you mentioned they are the same environments, so you can config one stage, then clone stages from the existing one, just name the stages with the specific App names accordingly. But you still need to change the settings accordingly, for example the Artifact filters (see Release triggers) to determine the condition to trigger the specific stage. But in this way it will download all the artifacts first when you add multiple artifact sources, this will take long time to get sources...
I'm new to Aure DevOps. Trying to create build and release pipelines there's one thing I don't understand:
Commonly, every kind of build finally results in some output, called artifacts.
With Azure DevOps it seems like there is always a final copy or publish task necessary to copy the created artifact from A to B, so the release task may then access the compiled artifacts.
Why aren't these artifacts plain accessible to a release pipeline right from the location where they have been built? Why don't the build tasks automatically set a variable pointing to the right folder, so the release pipeline may access the files right from there?
Or is this already happening and I'm just missing something from the tutorials I watched?
There are so many reasons.
Two easy ones:
There is no guarantee that the agent's working folder still contains the files. Agents are reused from build to build and release to release, and a given build or release will always use the same working folder. The working folder is cleaned up between builds.
Releases may run on different agents. On different machines. In different domains. Or any combination. There's no guarantee that the agent where the build ran is accessible by the agent where the release is running. Publishing the artifact allows a guarantee: As long as the machine the release is running on has the ability to talk to Azure DevOps (which is a requirement for the agent to function in the first place), it can get the artifacts it needs.
Why aren't these artifacts plain accessible to a release pipeline
right from the location where they have been built?
Agree with Daniel.
The main reason for me is because we can't hold the hosted agent all the time. Since MS wants to protect resources efficiently, it is not occupied for a long time.
When we queue a build, MS will assign us a brand new clean agent to execute our task, and after the build is complete, the MS will reclaim the agent assigned to our build and restore the agent to its initial state in preparation for accepting the assignment of the next task.
So, we could not keep hold the hosted agent to use it in next release pipeline. We have to store the artifacts in the cloud/server, then we could download it in the release pipeline. Otherwise, we could not get the artifact we need from an agent that has been restored.
Besides, MS is randomly assigned to the agent, and we cannot guarantee that the same agent will be allocated and built during the release pipeline.
That is the main reason why we need to copy or publish the artifacts.
If you do not want to copy or publish the artifacts, you could setup your own private agent, and do not clean the agent before you execute the release pipeline.
Update:
why is the user, well, bothered to find a place for the artifacts
manually? I would have expected every build pipeline to come with a
personal space to store the latest build artifacts. A space where
Azure DevOps automatically copies the build artifacts to. To me it
looks like things have to be manually copied from A to B and then
later from B to C.
That because not all output is needed, for example, the test project, what we need is test result/Code coverage for the test project, not the output for the solution. In this case, we do not need to copy the output to the artifacts. On the other hand, we need to copy some special files to artfacts, then automatically copy the build artifacts will not meet your requirements.
That is also the reason why we provide the task to copy files to artifacts, so that we could customize our personality needs.
Of course, if you think that manual copying is superfluous, you can use the MSBuild parameter /p:output=$(build.artifactstagingdirectory) to set the output directly to artifacts.
If I need to copy things from A to B in the Build pipeline, then what
should keep me from copying it to C right away? Then a separate
Release pipeline would be, well, rather optional, if not redundant.
If you are in the build pipeline, there is another task Download build artifacts, which could download the build artifacts.
if you are in the release pipeline, you just need select the build artifacts as source, release pipeline will download that artifact automatically:
Check this document for some more details.
Hope this helps.
I currently have a release defined for pushing clients to an internal nuget server.
Each release is identical in terms of steps, the only thing that changes is the triggering artifact.
However when I add another artifact to a "generic" release, so that there are now ClientA and ClientB artifacts on the release, both artifacts are downloaded when the release is triggered and they are both then pushed.
What I would like to know is if it is possible to have the release only download the triggering artifact.
Example:
A release is triggered on both ClientA and ClientB.
ClientB is released. The build triggers and only uses the ClientB artifact during the deployment.
I don't think this is possible at the moment, but something similar is possible though.
Create a task group from the current release definition as described here: https://learn.microsoft.com/en-us/vsts/pipelines/library/task-groups?view=vsts
There clone the release definition and change the artifact from ClientA to ClientB.
You'll end up having two release definition, both sharing the steps but the artifacts will be different.
EDIT: PowerShell script to download Build artifacts
Eddie suggested a nice workaround. Here's how you can pull down an artifact manually.
You need to enable Allow scripts to access OAuth token
Add a PowerShell task to each phase and reference the DownloadBuildArtifacts.ps1 script
If the release is triggered by build artifacts, the value of the pre-defined variable "$(RELEASE.TRIGGERINGARTIFACT.ALIAS)" will be the triggered artifacts alias. You can get the value of this variable to determine which artifact trigger the release.
There isn't any way to achieve this out of box. But you can disable auto download artifacts and add a powershell script in your release definition to get the triggered artifact and download it.
There isn't the way to just download the triggered artifacts, but you can identify which artifact trigger this release through RELEASE_TRIGGERINGARTIFACT_ALIAS variable.
So, you can push corresponding one per to that variable.