I have been searching and have not been able to find a solution to my issue of being able to run a task prior to the build being deleted by the retention policy of the project in VSTS. Here's my current setup:
Build runs, uploads artifacts to Artifactory.
The URL to the artifact is stored in the properties under build summary.
Project retention policy runs and removes builds that meet criteria.
Corresponding artifact in Artifactory remains.
What I want to do is, up on deletion of the build in VSTS, that somehow attach to the pre-deletion of the build and run a task to delete the corresponding artifact in Artifactory. Then continue to delete the build in VSTS.
Is this possible? Is there something I was missing when trying to search for this?
For this issue, from my point of view, it is impossible to remove external artifacts pre-deletion of build by project retention policy.
Because your retention policies are processed once per day. The timing of this process varies because we spread the work throughout the day for load balancing purposes. There is no option to change this process. So we can't track when the builds were deleted and cannot delete the corresponding external artifacts in the pre-deletion of the builds. For details ,please refer to this document.
You can through Artifactory Discard Builds task to remove build artifacts stored in Artifactory. Check the Delete artifacts checkbox, to also delete the build artifacts and not only the build meta-data.
Related
I have a bunch of Azure Pipeline and Release packages. The piplines build and publish my code to a testing environment every night, and as part of that publish they create artifacts used by the selenium testing software.
This has been working correctly for several years now. Sometime around the last upgrade we did to Azure Devops the artifacts have started getting deleted after about a day instead of honoring the deletion schedule that I have setup.
The artifacts are generated by the pipeline packages and used by the release packages, and everything is working correctly. However sometime after the last release has finished running for the night, all the artifacts get deleted. I have tried running the entire process manually, and when it is done the artifacts still exist (at least until the next day - I haven't identified a specific time they get deleted), so the issue does not appear to be within any of the packages themselves.
The Settings Retention policy, I believe these apply to the pipelines.
The Release Retention policy, obviously these should apply to the release packages.
Does anyone have any idea why my artifacts are not sticking around past 1 day?
It seems I have figured out the problem, and it appears to be a change in the way Azure Devops works with the newest version. This is the first version I have noticed this issue in:
Azure DevOps Server 2020 Update 1.1
The settings indicate that artifacts should be kept for 20 days in my instance. However, every time I run a build it over-writes the existing artifacts with the newest ones. My guess is that this artifact path gets tied to the build in the database. And since all artifact paths are identical in my case (something that did not occur to me to mention in the original post), deleting an artifact that is 20 days old will delete the data from the current artifact path as well.
I solved this issue for myself by including $(Build.BuildNumber) as part of my artifact path. Unfortunately you cannot directly use a date variable in that path, however since I have the date built into my buildnumber this approach worked for me.
The current scenario is I want a Release Pipeline that has multiple artifacts (one function and several apis). Each one with a cd trigger except for the one to prod which is manual for right now.
In the pipeline is there a way to prevent a particular job from running if deployed pipeline is the same as the original?
For example, a change is made to one of the apis but the rest remain the same I would want the release to get created and deployed but only run the deployment job linked to that particular artifact.
The reason being is that for a deployment to production it would be nice to have all the artifacts I'm looking to deploy in one place and all I would have to do to deploy to prod would be hitting deploy and if the latest artifact is newer then the one deployed it gets deployed but if it is not that deployment would be skipped.
Right now each artifact is in a separate release pipeline and each gets deployed separately, which means we are keeping a separate document with a list of all the release pipelines that needing to be deployed for a specific set of changes.
Any direction would be appreciated, thanks in advance.
I think you can do that with multi-stages and triggers. You can follow these steps:
Create a release pipeline.
Add all your artifacts.
Create multiple stages. According to your requirement, you may need as many stages as your artifacts.
Set triggers for each stage. Click on the lightning icon to the left of the stage. In "Triggers", select "Afer release". Then enable the "Artifact filters". Choose the artifacts that correspond to this stage as the trigger. Here is an example:
Then your release pipeline will be look like this:
When an artifact changes, the corresponding stages will be triggered, and no other stages will run.
It doesn't work. If an artifact is changed it doesnt checked that the artifact is newer
I am trying to create a Releasing stage that is meant to download 3 different build artifacts from 3 different pipelines in a project and deploy them on a target machine. When I create a release and deploy it on the target machine, the very first download build artifact task works fine but none of the following ones and I see the following error
"Artifact [buildnumber] not found for build [buildId]. Please ensure you have published artifacts in any previous phases of the current build."
Has anyone else seen this error or know why I am getting this error?
Thank
If you want to use Download Build Artifacts task to download different build artifacts from different pipelines, please select the “Specific build” option to find that particular artifact, as below.
If you use the default value: Current build, of course this task cannot find other build artifacts from other build pipelines. And then you will encounter this issue.
In addition, the new release will automatically download artifacts in below pre-defined “Download Artifacts” step.
Therefore, you could just add these 3 artifacts in release pipeline, and then you don’t need to use additional Download Build Artifacts task to download these artifacts. See: Release artifacts and artifact sources for details.
I'm new to Aure DevOps. Trying to create build and release pipelines there's one thing I don't understand:
Commonly, every kind of build finally results in some output, called artifacts.
With Azure DevOps it seems like there is always a final copy or publish task necessary to copy the created artifact from A to B, so the release task may then access the compiled artifacts.
Why aren't these artifacts plain accessible to a release pipeline right from the location where they have been built? Why don't the build tasks automatically set a variable pointing to the right folder, so the release pipeline may access the files right from there?
Or is this already happening and I'm just missing something from the tutorials I watched?
There are so many reasons.
Two easy ones:
There is no guarantee that the agent's working folder still contains the files. Agents are reused from build to build and release to release, and a given build or release will always use the same working folder. The working folder is cleaned up between builds.
Releases may run on different agents. On different machines. In different domains. Or any combination. There's no guarantee that the agent where the build ran is accessible by the agent where the release is running. Publishing the artifact allows a guarantee: As long as the machine the release is running on has the ability to talk to Azure DevOps (which is a requirement for the agent to function in the first place), it can get the artifacts it needs.
Why aren't these artifacts plain accessible to a release pipeline
right from the location where they have been built?
Agree with Daniel.
The main reason for me is because we can't hold the hosted agent all the time. Since MS wants to protect resources efficiently, it is not occupied for a long time.
When we queue a build, MS will assign us a brand new clean agent to execute our task, and after the build is complete, the MS will reclaim the agent assigned to our build and restore the agent to its initial state in preparation for accepting the assignment of the next task.
So, we could not keep hold the hosted agent to use it in next release pipeline. We have to store the artifacts in the cloud/server, then we could download it in the release pipeline. Otherwise, we could not get the artifact we need from an agent that has been restored.
Besides, MS is randomly assigned to the agent, and we cannot guarantee that the same agent will be allocated and built during the release pipeline.
That is the main reason why we need to copy or publish the artifacts.
If you do not want to copy or publish the artifacts, you could setup your own private agent, and do not clean the agent before you execute the release pipeline.
Update:
why is the user, well, bothered to find a place for the artifacts
manually? I would have expected every build pipeline to come with a
personal space to store the latest build artifacts. A space where
Azure DevOps automatically copies the build artifacts to. To me it
looks like things have to be manually copied from A to B and then
later from B to C.
That because not all output is needed, for example, the test project, what we need is test result/Code coverage for the test project, not the output for the solution. In this case, we do not need to copy the output to the artifacts. On the other hand, we need to copy some special files to artfacts, then automatically copy the build artifacts will not meet your requirements.
That is also the reason why we provide the task to copy files to artifacts, so that we could customize our personality needs.
Of course, if you think that manual copying is superfluous, you can use the MSBuild parameter /p:output=$(build.artifactstagingdirectory) to set the output directly to artifacts.
If I need to copy things from A to B in the Build pipeline, then what
should keep me from copying it to C right away? Then a separate
Release pipeline would be, well, rather optional, if not redundant.
If you are in the build pipeline, there is another task Download build artifacts, which could download the build artifacts.
if you are in the release pipeline, you just need select the build artifacts as source, release pipeline will download that artifact automatically:
Check this document for some more details.
Hope this helps.
Last week, our team decided to move all separate .NET projects into a single solution. Because we were already using Azure DevOps with separate build pipelines for each project, we looked into the possibility to only trigger builds whenever there are changes to a specific project. This, to prevent the build agents from rebuilding the entire solution on every commit.
Because our source repository is a locally hosted Git one, we do not have the niceties of using path filters, so we resorted to using a Powershell task to determine if all upcoming tasks should be skipped, using a custom condition (explained in this SO question).
Now, on every commit, all builds are triggered, but the builds resolve for themselves if they should continue to build and (more importantly) if they should publish a build artifact at the end. An example of such a skipped build is below:
However, every time such a build 'succeeds', a new artifact is created, even though the publish artifact task is skipped. The problem is that all our release pipelines are triggered when a build pipeline creates an artifact. So every time a build is skipped like shown above, the release pipeline triggers and tries to deploy the artifact. But it fails, because the artifact it uses to deploy does not really exist (it seems like it is empty, and the build pipeline actually doesn't mention an artifact on the Summary tab), so any operations on the artifact fail in the release pipeline.
Our release pipelines are set to trigger on source type build artifact, for the corresponding build pipeline, with the default version parameter set to latest.
How is it possible that, even though the publish artifact task is skipped, the release pipeline still triggers and tries to deploy an empty artifact?
I assuming you enable the Continues Delivery in the Releases pipelines, this option is not related to build artifacts. this option means that Release will be triggered when the build succeeded (not matter if he has artifacts or no).
So this is the reason why after each build a new release started.
As workaround, you can add a task in the build that add "build tag" only when there is a artifacts, and in the release artifacts options, instead of choose Latest you can choose Latest from the build pipeline default branch with tags and specify the tag you put in the build.
Another option is in the "Stages" click on the triggers and then it's configured to "After release" so enable the "Artifact filters" and specify there the build tag.
How do you add a build tag? add a PowerShell task with this command:
Write-Host "##vso[build.addbuildtag]test-tag"