We're building out a release pipeline in Azure DevOps which pushes to a Kubernetes cluster. The first step in the pipeline is to run an Azure CLI script which sets up all the resources - this is an idempotent script so we can run it each time we run the release pipeline. Our intention is to have a standardised release pipeline which we can run against several clusters, existing and new.
The final step in the pipeline is to run the Kubectl task with the apply command.
However, this pipeline task requires specifying in advance (at the time of building the pipeline) the names of the resource group and cluster against which it should be executed. But the point of the idempotent script in the first step is to ensure that the resources and to create if not.
So there's the possibility that neither the resource group nor the cluster will exist before the pipeline is run.
How can I achieve this in a DevOps pipeline if the Kubectl task requires a resource group and a cluster to be specified at design time?
This Kubectl task works with service connection type: Azure Resource Manager. And it requires to select Resource group field and Kubernetes cluster field after you select the Azure subscription, as below.
After testing, we find that these 2 fields supports variable. Thus you could use variable in these 2 fields, and using PowerShell task to set variable value before this Kubectl task. See: Set variables in scripts for details.
Related
I have multiple local on-prem agents that can run a particular deployment group job (for the purpose of load balancing). I want only the first available one to run the job and not all of them. Which setting of the Deployment Group Job I can use to do that? My only options seems to be "Multiple" and "Single at a time" both of which run the jobs on all servers matching the Required Tags.
I want only the first available one to run the job and not all of them. Which setting of the Deployment Group Job I can use to do that?
There is an option for the Release pipeline with Deployment Group job Required tags：
Then we just need to add the tag to the machine, which we want to deploy:
Now, the release pipeline will run a Deployment Group Job in one of the matching servers.
Update:
I do have the required tag, but on two agent servers. I want the Job
to pick only one of the two matching servers and run the job on it
instead of running it on both.
As workaround, you could create a private agent pool and add two or more agents in it. Those agents are deployed on different machines. With this way, the pipeline will executed on one of the agent.
https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops&viewFallbackFrom=vsts
I am working with 9 multiple repositories with their own separate azure pipelines. However, during the deployment stage, there are some jobs/steps that are dependent on the status of a job from a separate deployment pipeline. If the resource (e.g A Resource Group) was not yet deployed, naturally the job/step dependent on it would fail. I cannot simply use dependsOn because it is from another pipeline.
Questions:
Is there a way for me to check, monitor, or wait (with a specified period of time) for a particular resource to be deployed?
Are these pipelines in need of restructuring?
I've seen documentation for the Multi-Repo pipeline. However, each pipeline must be deployable on its own and not waiting for another pipeline to finish.
If you use YAML based pipeline you may consider pipeline triggers. It will allow you to trigger one pipeline after another. You may also apply stages filter
In this sprint, we added support for 'stages' as a filter for pipeline resources in YAML. With this filter, you don't need to wait for the entire CI pipeline to be completed to trigger your CD pipeline. You can now choose to trigger your CD pipeline upon completion of a specific stage in your CI pipeline.
I'm using Azure Pipeline Agents on Machines and have those Machines in a Deployment Group and I have a DevOps Release which does some things on each machine. If the Azure Pipeline Agent isn't running on a machine at release time, the release will skip over this machine (see below image). How can I know which machines were skipped?
!]1
How can I know which machines were skipped?
The easiest way to check is that you can manually check the detailed deployment log.
For example:
Then you could get the skipped agent name.
On the other hand, you could also use the Rest API : Releases - Get Release. In the API response, you could check the Job Status and the Agent name.
Here is sample:
GET https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/releases/{releaseId}?api-version=6.0
What are the differences between an Agent Job and a Deployment Group Job in Azure DevOps? What are the reason to create one or the other?
What are the differences between an Agent Job and a Deployment Group
Job in Azure DevOps?
Agent job:
Run steps on an agent which in an agent pool.
Deployment group jobs:
Run on machines in a deployment group.
These are the definition of them. You can see that, the fundamental difference between them is that the target when running the job are different.
For agent job, it can only run on one target at a time (unless set up parallel to run on multiple targets at a time, but parallel is essentially multiple jobs). And deployment group job is, since the deployment group is multiple machines are bound in a group, it can run a job on multiple machines at a time.
In the usage scenario, Agent job can used in both Build and Release pipeline. But for Deployment agent job, it can only be used in Release pipeline for application/project deploy.
What are the reason to create one or the other?
In build pipeline, it should no doubt that you can only use Agent job (or Agentless) job.
I think what you concerned should be the usage in Release pipeline. As I mentioned above, these different jobs can all be used in Release pipeline, and they can all be used for project deployed.
But in terms of specific use, it depends on the task you will use and the number of target servers you want to deploy to.
Agent job:
If your deployment target server number less than 5 objects, and need to deploy to multiple machines at the same time, you can set up a parallel job for Agent job. The Agent job may take a little longer time than Deployment group job. But because the number of deployed targets is not too much, the difference is not obvious.
Deployment group job:
For medium and large companies, the Deployment target objects are generally more than 10, even 100. It is most appropriate to use Deployment group job, because it can be deployed on different machines in one job.
In release, recommend you use Deployment group job if you have multiple targets to deploy to:
I'm trying to re-use a Deployment Group Job and execute it for each member of a ClientID list, but it is not working. The name of the stage will also be used in my tasks.
As described on this page...
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=classic
I've create two variables:
ClientID
client1, client2, client3
Multipliers
ClientID
The stage only has a Deployment Group Job and not an Agent Job because after the deploy I have tasks to run on the self-hosted servers. I need to stop and remove a service, copy files, register and restart a service.
There are no other options on the Stage or the Deployment Group Job for multipliers. (There is the "multiple" options for parallel deployments to servers within the group but that is unrelated as far as I know.)
If I add an Agent Job then I can see that it has the "Multi-configuration" and "Multipliers" options that other articles mention. On a deploy the agent job executes multiple times as expected but not the deployment group job.
The article above talks about "Multi-configuration deployments", so I'm I wrong to think the Deployment Group Job should work that way?
Any help is appreciated. Thanks.
I am afraid that using multipliers in the deployment groups is not feasible.
According to the definition of Multi-configuration in the agent job : each configuration will use a single agent.
But, in effect, a deployment group is just another grouping of agents, much like an agent pool.It could not assign a single agent to each configuration.
So I think that multipliers are not available in the deployment groups.If you want to use multipliers, then you need to use agent job.