Is there any tool can monitor helm release ?
You may come up with this one https://github.com/ContainerSolutions/helm-monitor, but i don't want automatic rollback by plugin. This is intrusive to our ci/cd deployment.
what i need.
1. i can integrate with grafana to see a history of the release and also its status in history
2. I can set up alerts when some release is stuck in pending or failed then i can notify related teams
Related
I have a pipeline that is automatically building and deploying code to my staging environment. For my production environment I have a pre-deployment manual approval gate so that only releases that have gone through some review will go out to customers. So far so good.
The problem is that as new releases go out to the staging environment there is a growing list of releases that are now queued for this manual approval. In order to release the most recent version I need to go and manually reject each of the intermediate releases. This has become a laborious process.
I would like to automatically reject the production deploy of the previous release every time a new release goes to staging.
I've looked at the MS docs, SO, the pipeline settings, the available pipeline release tasks and can't find a way to do this.
Release History showing old release queued for approval:
Looks like this behavior can be controlled by making a change in the Deployment Queue Settings area. Switching to "Deploy latest and cancel the others" will automatically cancel the previously queued release and queue the newer one. If you have Slack integration turned on (as I do) you will see a cancellation message.
So long as your process is simple enough that you know when new builds are being made, this feels like good behavior. It gives you a basic manual gate without adding any other overhead.
There is more documentation here: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/stages?view=azure-devops&tabs=classic#queuing-policies
Deployment Queue Setting: subsequent releases
We are using Azure CI/CD pipeline and have started publishing to deployment slots. After doing this change the publishing history is no longer updated, not in the production or staging slot. Is there any way to keep track on whats in production when using deployment slots?
I'm trying to configure Azure DevOps Release pipelines for our projects, and I have a pretty clear picture of what I want to achieve, but I'm only getting almost all the way there.
Here's what I'd like:
The build pipeline for each respective project outputs, as artifacts, all the things needed to deploy that version into any environment.
The release pipeline automatically deploys to the first environment ("dev" in our case) on each successful build, including PR builds.
For each successive environment, the release must have been deployed successfully to all previous environments. In other words, in order to deploy to the second environment ("st") it must have been deployed to the first one ("dev"), and in order to deploy to the third ("at") it must have been successfully deployed to all previous (both "dev" and "st"), etc.
All environments can have specific requirements on from what branches deployable artifacts must have been built; e.g. only artifacts built from master can be deployed to "at" and "prod".
Each successive deploy to any environment after the first one is triggered manually, by someone on a list of approvers. The list of approvers differs between environments.
The only way I've found to sort-of get all of the above working at the same time, is to automatically trigger the next environment after a successful deployment, and add a pre-deployment gate with a manual approval step. This works, except the manual approval doesn't trigger the deployment per se, but rather let an already triggered deployment start executing. This means that any release that's not approved for lifting into the next environment, is left hanging until manually dismissed.
I can avoid that by having a manual trigger instead of automatic, but then I can't enforce the flow from one environment to the next (it's e.g. possible to deploy to "prod" without waiting for successful deployments to the previous stages).
Is there any way to configure Azure DevOps Release Pipelines to do all of the things I've outlined above at once?
I think you are correct, you can only achieve that by setting automatic releases after successful release with approval gates. I dont see any other options with currect Azure DevOps capabilities.
Manual with approval gates doesnt check previous environments were successfully deployed to, unfortunately.
I hope this provides some clarity after the fact. Have you looked at YAML Pipelines In this you can specify the conditions on each stage
The stages can then have approvals on them as well.
Quick question.
Is there a way to constrain/restrict what order users can can deploy builds to environments?
For example if I have these four environments configured with manual push-button deploy (not-automated) I can start all four together if I want. I don't have to wait for the other to be done before kicking off the next one:
DEV
TEST
STAGE
PROD
Microsoft seems to be missing this feature in TFS 2015. It would make sense to offer a deployment condition that states that previous environments must have successful deployments before you can run push-button deploy for the next.
Yes, I know, you are going to say "but you can automate that so the deploys run in the order you want." Management here does NOT want that. They want push button deployment for each environment WITH a constraint that previous environments must be completed first.
This means a manual start for each environment.
Other than having the release manager "eyeball" the situation before pushing the button for the next environment I can't see a way to configure this rule.
Any ideas?
There is not any restriction on manually deploy situation for now. This is designed for giving you the ability to override the release process.
Note that you can always deploy a release directly to any of the
environments in your release definition by selecting the Deploy
action when you create a new release.
In this case, the environment triggers you configure, such as a
trigger on successful deployment to another environment, do not apply.
The deployment occurs irrespective of these settings. This gives you the ability to override the release process. Performing such
direct deployments requires the Manage deployments permission, which
should only be given to selected and approved users.
Source Link: Environment triggers
Suggest you use automation triggers, you could use Parallel forked and joined deployments, in combination with the ability to define pre- and post-deployment approvals, this enables the configuration of complex and fully managed deployment pipelines to suit almost any release scenario.
If you insist on manual push-button deploy, you may have to ask the release manager "eyeball" the situation to restrict environment deployment order as you mentioned.
I have a problem in Release Management 2013. A release was failing in its deploy actions, so it was stopped, giving it status Cancelled. Then the release was abandoned. However, still two action are still In Progress, after two days now. I have full rights to edit and manage all parts of this release, but now unable to stop, delete or retry this release.
New releases for the same target server give the following error:
New deployment is not allowed as another deployment is in progress.
Retry the deployment after sometime.
Release Management Monitor service was restarted but this didn't help.
Release are based on vNext templates
Anybody idea how to resolve this?
There is an option to force reject a run-away release in Release Management 2015.