Azure Devops deployment failure: The 'Performing deployment' operation conflicts with the pending 'Performing deployment' operation - azure-devops

I am trying to create a devops pipeline to deploy an azure function.
Each time I try I get the error:
BadRequest: The 'Performing deployment' operation conflicts with the
pending 'Performing deployment' operation started at 2022-08-16T13:01:47.6881726Z.
Please retry operation later.
I have waited 2 hours and still get this error.
In the resource group, i cannot see any pending deployments, only failed deployments.
Also, get-AzDeployment cmdlet returns no data so i cant find any deployments that may be blocking.
Any ideas how to resolve this?

The message usually indicates that there is another pending deployment operation that is ongoing and it would prevent the new deployment. However, you don't see any results from running the get-AzDeployment cmdlet. Additionally, you can run the Get-AzDeploymentOperation cmdlet as it lists all the operations that were part of a deployment to help you identify and give more information about the exact operations that failed for a particular deployment.
https://learn.microsoft.com/en-us/powershell/module/az.resources/get-azdeploymentoperation?view=azps-8.3.0

I found the issue was due to exporting the arm template from the portal.
For some reason the portal included a bunch of "deployments" in the template. These deployments which had already run were clashing when I ran the template.
I've no idea why it includes old deployments in the template but deleting them resolved that particular issue.

Related

Azure DevOps Server - multistage YAML pipeline - release couldn't start

On my Azure DevOps Server instance (2020 Update 1.1) I have easy multistage YAML pipeline with Build job (run against BuildPool) and release job (run against ReleasePool). Build job is executed successfully. In release pool there are many idle agents but job is in waiting state with message:
The agent request is not running because all potential agents are running other requests. Current position in queue: 1
No agents in pool ReleasePool are currently able to service this request.
Other pipelines on the server against ReleasePool are executed.
This pipeline was executed one month past also successfully, and since this execution the YAML definition stays unchanged.
Pipeline have no explicit demands, I'm trying to identify implicit demands (from used tasks - I have checked tasks.json task manifests for each used task) - but there isn't used no task with demands.
I have no idea what I could try next.
Is the way how to diagnostic how are agents assigned to the pipeline jobs? I have admin permissions and access to the DB, I'm ready to do very deep analysis.
Since other tasks are running fine, maybe a specific demands on selected agents are not met and this is preventing an agent to be found.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/demands?view=azure-devops-2020&tabs=yaml
Or else, it's a bug.
I found a similar issue here, and Microsoft responded July 2022 with:
"We have released a fix for this issue"
https://developercommunity.visualstudio.com/t/no-agent-found-in-pool-azure-pipelines/870000
It is however not clear to me if this only applied to Azure DevOps or also to Azure DevOps Server.
But since your on 2020 Update 1.1, updating couldn't harm you:
https://learn.microsoft.com/en-us/azure/devops/server/release-notes/azuredevops2020u1?view=azure-devops-2020

Deployment started failing - decryption operation failed

Deployment task of a pipeline has been working fine until yesterday. No changes that Im aware of. This is for a build that uses a deployment agent on an on-prem target. Not sure where to look, other than possibly reinstall the build agent via the script?
2021-08-11T18:36:20.7233450Z ##[warning]Failed to download task 'DownloadBuildArtifacts'. Error The decryption operation failed, see inner exception.
2021-08-11T18:36:20.7243097Z ##[warning]Inner Exception: {ex.InnerException.Message}
2021-08-11T18:36:20.7247393Z ##[warning]Back off 28.375 seconds before retry.
2021-08-11T18:36:51.6834124Z ##[error]The decryption operation failed, see inner exception.
Plese check announcement here
The updated implementation of BuildArtifact tasks requires an agent upgrade, which should happen automatically unless automatic upgrades have been specifically disabled or the firewalls are incorrectly configured.
If your agents are running in firewalled environments that did not follow the linked instructions, they may see failures when updating the agent or in the PublishBuildArtifacts or DownloadBuildArtifacts tasks until the firewall configuration is corrected.
A common symptom of this problem are sudden errors relating to ssl handshakes or artifact download failures, generally on deployment pools targeted by Release Management definitions. Alternatively, if agent upgrades have been blocked, you might observe that releases are waiting for an agent in the pool that never arrives, or that agents go offline half-way through their update (this latter is related to environments that erroneously block the agent CDN).
To fix that, please update your self hosted agents.

Conditionally run an Azure DevOps job only if a specific prior job has failed

In Azure DevOps, I'm trying to make it so that a release job (not a YAML pipeline job) only runs if a specific prior job has failed. One of the pre-defined conditions for running a job is "Only when a previous job has failed", but this is not appropriate because it includes all prior jobs, rather than just the last job (or better yet, a job of the users choosing).
Please note that this question focuses on tasks within a job and is not the answer that I am looking for.
According to the documentation for conditions under "How can I trigger a job if a previous job succeeded with issues?", I can access the result of a previous job -
eq(dependencies.A.result,'SucceededWithIssues')
Looking at the logs for a previous release, the AGENT_JOBNAME for the job that I wish to check is Post Deployment Tests, so that would mean that my condition should look like this. However, Azure DevOps wont even let me save my release -
not(eq(dependencies.Post Deployment Tests.result, 'succeeded'))
Job condition for job "Swap Slots Back on Post Deployment Test Failure" in stage "Dev" is invalid: Unexpected symbol: 'Deployment'.
I've tried to wrap the job name in quotes, but I get similar errors -
not(eq(dependencies.'Post Deployment Tests'.result, 'succeeded'))
not(eq(dependencies."Post Deployment Tests".result, 'succeeded'))
I've also tried to reference my job using underscores, which does allow me to save the release but then results in an error at run time -
not(eq(dependencies.Post_Deployment_Tests.result, 'succeeded'))
Unrecognized value: 'dependencies'.
How can I achieve me goal of conditionally running a job only if a specific prior job has failed?
How can I achieve me goal of conditionally running a job only if a
specific prior job has failed?
1.You should know that Yaml pipeline and Classic pipeline use difference technologies.
See feature ability, they have different features. Also, they have quite different behavior for Conditions though they(Yaml,Classic Build,Classic Release) all support job conditions.
The eq(dependencies.'Post Deployment Tests'.result, 'succeeded') you're trying is not supported in Classic(UI) Release pipeline. It's for Yaml:
It's expected behavior to get error like Unrecognized value: 'dependencies' cause the job dependency is not supported in Release pipeline. See this old ticket.

I'm having issues with DevOps production deployment - Unable to edit or replace deployment

Up until yesterday morning I was able to deploy data factory v2 changes in my release pipeline. Then last night during deployment I received an error that the connection was forced closed. Now when I try to deploy to the production environment, I get this error: "Unable to edit or replace deployment 'ArmTemplate_18': previous deployment from '12/10/2019 10:19:27 PM' is still active (expiration time is '12/17/2019 10:19:23 PM')". Am I supposed to wait a week for this error to clear itself?
This message indicates that there’s another deployment going on, with the same name, in the same ARM Resource Group. In order to perform your new deployment, you’ll need to either:
Wait for the existing deployment to complete
Stop the in-progress / active deployment
You can stop an active deployment by using the Stop-AzureRmResourceGroupDeployment PowerShell command or the azure group deployment stop command in the xPlat CLI tool. Please refer to this case.
Or you can open target Resource Group on the azure portal, go to Deployment tab, find not completed deployments, cancel it, start new deploy. You can refer to this issue for details.
In addition, there is a recently event of availability degradation of Azure DevOps .This could also have an impact. Now the engineers have mitigated this event.

Azure DevOps: How's the deployment group job status decided?

I have a deployment group job with setting in below pic as 33%. How's the status of job determined to be "failed" or partially succeeded"? The description of the setting only specifies "It is also used to determine the success and failure conditions during deployment" which is not really clarifying.
In first attempt, status is failed even though only 4 machines didn't succeed
After couple of attempts, status is "partially succeeded" even though 23 failed
I think you should have been know that this does not has any relevant with the number of the failed deployment.
In fact, this only relevant with one option: Continue on error
When you enable this option in the task, even one step failed, it will still continue the deploying. At last, it will display partially succeeded.
Instead, if you did not check this Continue on error option, even it only has one failed step, the status will still display failed.
To verify this, you can check your task configuration which exists in Control Options.
Also, you can check this thread: VSTS Release - Phase with partially succeeded.
Deployment group phase will be “partially succeeded” if deployment is
attempted to all the targets, event the deployment is failed in any of
the target.
I found the answer from the link in Merlin's answer below : "Deployment group phase will be “partially succeeded” if deployment is attempted to all the targets, event the deployment is failed in any of the target."
Couldn't locate this piece of info anywhere in the official docs.