Azure Pipelines how filter artifacts per stage for "Manual only" triggered Releases - azure-devops

Let's say I have these 3 Stages: Dev, QC, Prod.
My requirements are:
Artifacts only from specific branches(release/*) can be deployed to QC/Prod
Artifacts from all branches can be deployed to Dev
I can achieve what I want using Artifact filters for "After stage" triggered Releases but I need this for "Manual only".
Is there a workaround that will let me control/filter which artifacts are available for deployment for specific stages/environments?
Basically, I need the Azure DevOps equivalent of Octopus Channels.
Update
I think I'm close to a solution.
In the "Pre-deployment conditions", I can add a new Deployment Gate which makes a Rest API call.
e.g URL suffix=/Release/releases/76
Now, I just need to correctly parse the ApiResponse because the below Success criteria doesn't work
eq(root['artifacts[0].definitionReference.branch.id'], 'refs/heads/master')
Evaluation of expression 'eq(root['artifacts[0].definitionReference.branch.id'], 'refs/heads/master')' failed.

As you said, you can do this using Deployment gates on your stages.
Create a new Generic service connection from Project Settings -> Pipelines -> Service Connections.
For service URL something like https://vsrm.dev.azure.com/{OrgName}/{ProjectName}/_apis
On your stage, open the Pre-Deployment Conditions
Enable the Gates option.
Add a new Invoke REST API gate and set the Delay before evaluation to 0 minutes.
4.1 Set the connection type to Generic.
4.2 Select the service connection you created in step 1.
4.3 Set the method to GET.
4.4 Set the URL suffix to /Release/releases/$(Release.ReleaseId)
4.5 On the Advanced area, set the Completion Event to ApiResponse.
4.6 On the Advanced area, set the success criteria to (or startsWith)
eq(root['artifacts'][0]['definitionReference']['branch']['id'],'refs/heads/master')
Now, if you try to deploy an artifact not from the master branch, the deployment will fail

There is a workaround:
In the QC/Prod stages add a custom condition that the job will be executed only where the artifacts source branch is release/*:
startsWith(variables['Release.Artifacts.{Artifacts-Alias}.SourceBranch'], 'refs/heads/release')
Now, when you manually run the QC/Prod stages and the artifacts not came from the release the job not will be executed:

This works
and(contains(variables['build.sourceBranch'], 'refs/heads/release'), succeeded())

Related

FileTransform Task in release pipeline doesn't update the value - AzureDevOps

I'm trying to update the value of appsettings.Development.json file while releasing application using Azure DevOps pipeline.
appSettings.Development.json
{
"Networks": {
"EnableNetwork": {
"SomeNetwork": {
"SomeValue": "Old Value"
}
}
}
}
I have configured File Transform: task as follows
And set the variable with scope "Release"
File Transfer Task runs successfully
However, it doesn't change the value after deployment. Tried different changes as shown in SO answers like How to change Appsettings and Config info in Release Pipeline. But not sure where is the issue!
The steps you use File Transform task should be correct. The value of appsettings.Development.json can be updated successfully.
You can check the log of File Transform task to confirm this point.
For example:
it doesn't change the value after deployment.
From your screenshot, the cause of this issue could be that you are using Microsoft-hosted agent to deploy the IIS WebSite. It will deploy the website to hosted agent instead of your local machine.
According to your screenshot of the release pipeline definition, the Stage 3 should be run on Deployment Group. In this case, it will deploy the package to local machine.
You need to check the release pipeline definition to make sure the stage is running on Deployment Group. Then you will see the changes after deployment.

Tag resources when registering to the environment

I have a pipeline with multiple stages that deploys groups of virtual machines.
And registers one to and azure pipelines environment.
Then I want to target that registered VM in a deployment job.
I have a problem to target that resource by name as the resource does not exists in the environment at queue time so I cannot even disable the stage before running.
So my next option is targeting by tags.
But I saw no option in the registration script to define tags at registering time.
So my pipeline flow has a manual step between stages to go to the environment and tag the resource.
Then I can trigger the deployments stage of the pipeline and it continues ok.
So my question is:
There is any way of disabling the resource evaluation at queuetime or anyway to tag resoureces in the environment programmatically?
Thanks
But I saw no option in the registration script to define tags at
registering time.
When running the Registration script, there will be a step: Enter Environment Virtual Machine resource tags? (Y/N) (press enter for N), at this time you need to enter Y, and then in the next step: Enter Comma separated list of tags (e.g web, db) define the tag for the resource.
Update:
You can add --addvirtualmachineresourcetags --virtualmachineresourcetags "<tag>" to the registration script.
You can refer to this case for details.

Azure devops release variables are not getting resolved in custom server task

Below Release Variables are not getting resolved at run time when tried in a custom server task on a Release Pipeline.
System.JobName
System.JobDisplayName
System.StageDisplayName
System.DefinitionName
I am able to fetch JobId by using $(system.JobId), but not other job details (JobName) as mentioned above.
Am I missing anything here:
Those YAML pipeline variables. They will not work in a classic release pipeline.
Use the release-specific equivalents: https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch#default-variables

Download artifact from Azure DevOps Pipeline grandparent Pipeline

Given 3 Azure DevOps Pipelines (more may exist), as follows:
Build, Unit Test, Publish Artifacts
Deploy Staging, Integration Test
Deploy Production, Smoke Test
How can I ensure Pipeline 3 downloads the specific artifacts published in Pipeline 1?
The challenge as I see it is that the Task DownloadPipelineArtifact#2 only offers a means to do this if the artifact came from the immediately preceding pipeline. By using the following Pipeline task:
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'specific'
project: '$(System.TeamProjectId)'
definition: 1
specificBuildWithTriggering: true
buildVersionToDownload: 'latest'
artifactName: 'example.zip'
This works fine for a parent "triggering pipeline", but not a grandparent. Instead it returns the error message:
Artifact example.zip was not found for build nnn.
where nnn is the run ID of the immediate predecessor, as though I had specified pipelineId: $(Build.TriggeredBy.BuildId). Effectively, Pipeline 3 attempts to retrieve the Pipeline 1 artifact from Pipeline 2. It would be nice if that definition: 1 line did something, but alas, it seems to do nothing when specificBuildWithTriggering: true is set.
Note that buildType: 'latest' isn't safe; it appears it permits publishing an untested artifact, if emitted from Pipeline 1 while Pipeline 2 is running.
There may be no way to accomplish this with the DownloadPipelineArtifact#2. It's hard to be sure because the documentation doesn't have much detail. Perhaps there's another reasonable way to accomplish this... I suppose publishing another copy of the artifact at each of the intervening pipelines, even the ones that don't use it, is one way, but not very reasonable. We could eliminate the ugly aspect of creating copies of the binaries, by instead publishing an artifact with the BuildId recorded in it, but we'd still have to retrieve it and republish it from every pipeline.
If there is a way to identify the original CI trigger, e.g. find the hash of the initiating GIT commit, I could use that to name and refer to the artifacts. Does Build.SourceVersion remain constant between triggered builds? Any other "Initiating ID" would work equally well.
You are welcome to comment on the example pipeline scenario, as I'm actually currently using it, but it isn't the point of my question. I think this problem is broadly applicable, as it will apply when building dependent packages, or for any other reasons for which "Triggers" are useful.
An MS representative suggested using REST for this. For example:
HTTP GET https://dev.azure.com/ORGNAME/PROJECTGUID/_apis/build/Builds/2536
-
{
"id": 2536,
"definition": {
"id": 17
},
"triggeredByBuild": {
"id": 2535,
"definition": {
"id": 10
}
}
}
By walking the parents, one could find the ancestor with the desired definition ID (e.g. 10). Then its run ID (e.g. 2535) could be used to download the artifact.
#merlin-liang-msft suggested a similar process for a different requirement from #sschmeck, and their answer has accompanying code.
There are extensions that allow you to do this, but the official solution it to use a multi-stage pipeline and not 3 independent pipelines.
One way is using release pipelines (you can't code/edit it in YAML) but you can use the same artifacts through whole deployment.
Release pipeline
You can also specify required triggers to start deployment on
Approval and triggers
Alternatively, there exist multi-stage pipeline, that are in preview.(https://devblogs.microsoft.com/devops/whats-new-with-azure-pipelines/ ).
You can access it by enabling it in your "preview feature".
Why don't you output some pipeline artifacts with meta info and concatenate these in the preceding pipes like.
Grandparent >meta about pipe
Parent > meta about pipe and grantparent meta
Etc

Notification when cant download artifact

We have a scheduled release in Octopus that deploys the last known good release to Prod back to Prod.
However this has started failing because the artifact has fallen out of our retention policy - this we can fix by altering the retention policy.
The real issue is that when it failed no notifications were sent to the team because artifact collection happens before even the first step.
I have tested this with a dummy release that just has a single basic step and then a Slack Notification step for when it fails. However, we never get to the first step - let alone the slack step.
How can i hook on to this failure so that we know about these issues in future.
You have to follow below steps to achieve the same
Step 1) Add Email Template step # First : to inform that Build is triggered
There is a setting in that called : Start Trigger set it to Run in parallel with the previous step so email will be triggered while your artifacts are downloading
Step 2) Add Email Template step # Last : to inform that build failed
Just change the setting Run Condition set it to : Failure: only run when a previous step failed
so when your deployment get fail, It will notify the same. You can add the cause of failure in email body using inbuilt variables also.