We are using octopus to deploy our project. A bunch of steps which gets executed during the deployment. One of them is a powershell script and that powershell script is a work in progress.
However to test the script we have to perform a dummy check in or can create a new release in octopus after we change the build powershell script step, and it will pick up the build steps straight away and does not cache, else the script which gets executed is the previous version.
I do not know if this is caching or some other issue. I think this is some kind of issue with octopus or setting which I am missing.
Please help.
An important aspect of deployment automation is ensuring that deployments are repeated exactly each time they run.
When you create a release in Octopus Deploy, the artefacts, process, and variables are all "locked in" for that release. This means no matter what changes you make, for the lifetime of that release it will be performed identically every time.
If your deployment tool didn't do this, the same relase could work in your test environment, but then fail in the live environment because the deployment process changed in some way.
In effect, you release changes to the deployment process in the same way you release changes to the application itself.
This is why you need to create a new release in Octopus Deploy in order to see the changes you make.
This is both a blessing and a curse... On the one hand - your existing release scheduled for Production is protected from changes being made in lower environments. On the other hand - you are forced to recreate a release if you need to make a slight process change mid-cycle. This is arguable the correct approach since you would want to test any changes - but maybe not relevant if your changes can only be tested in higher environments (e.g maybe only Production is load balanced).
The software does allow to update Variables mid-cycle, but not Process Steps. I believe this feature is been requested for a future release.
http://help.octopusdeploy.com/discussions/questions/5130-how-to-update-a-single-variable-in-an-existing-release
Related
My question is about maintainability of vNext/Octopus processes vs XAML based processes. Or rather about the impossibility to maintain them sanely leading me to believe we are doing something terribly wrong.
Given:
Microsoft pushes to phase out its TFS XAML builds in favour of the vNext builds
Octopus Deploy is a popular deployment automation framework
We have many XAML based builds, but starting to port to vNext
The deployments are automated with Octopus Deploy
Concretely, we have three kinds of builds going on in QA:
Old XAML based compilation builds producing artifacts to be deployed
Ultimately just builds the code, zips it and places in a well-known location
New vNext compilation builds producing artifacts to be deployed
Same as above
Deployment builds
XAML based build definition per deployment environment. This is the source of truth for the particular deployment, containing all the configuration URLs, connection strings, certificate thumbprints, etc.. The build definition has over 100 build parameters. Each time a new environment is setup we clone an existing XAML build definition and change the parameters.
This build unpacks the build artifact, generates all the web/app config files based on the configuration parameters and kicks off Octopus Deploy with a lot of parameters using octo.exe and waiting for the end
Octopus Deploy process
Creates 3 packages from the build artifact previously unpacked by the XAML build to match three areas of deployment - web farm, background job engine cluster and the database
Delivers the relevant packages to the relevant tentacles.
The tentacles unpack and setup their respective packages
So, if we have 50 deployment environments, then we have 50 XAML deployment builds, each capturing the context of the respective environment. But the XAML deployment build delegates the deployment job to Octopus and here we are forced to have 50 Octopus projects - one per deployment.
Why is it so? We examined the option of having just one Octopus Project, but what would be the Release versions of such project? In order for us to be able to navigate amongst the gazillion releases, the release version must include:
The build version of the deployed code, e.g. 55.0.18709.3
The name of the deployment environment, e.g. atwfm
Using the example above this gives us 55.0.18709.3-atwfm, but sometimes we want to deploy the same build artifact in the same deployment environment twice. But the only Octopus project would already have the release 55.0.18709.3-atwfm, so how to deploy 55.0.18709.3 in atwfm again, without deleting the already existing release?
We could not find a workaround and so, we have Octopus project per deployment.
THIS IS ABSOLUTELY CRAZY because Octopus projects are a pain to update. Suppose we need to add a step - go do it in 50 projects. There are great advises on the Internet to use automation to edit multiple projects. Not ideal at all.
vNext, BTW, has the same problem. If I am to port the existing XAML builds to vNext I will end up with 50 vNext deployment builds. If I decide to add a step, I need to do it in all the 50 builds!!!
Note, that XAML builds do not have this problem (they have many others, though), because their the process is separate from the parameters. I can modify the workflow once and all the XAML builds are now updated with the new process change.
My question is - how do people work with vNext and Octopus, because our process drives me crazy. There must be a better way.
EDIT 1
I would like to clarify. We sometimes want to deploy the same build artifacts twice. We are not recompiling them and reusing the same version. No. We already have the build artifacts handy with the build version baked inside the artifact. We may want to deploy it the second time into the same environment because, for example, some databases in that environment have been misconfigured and now this is fixed and we need to redeploy. This does not mean we can rerun the already existing Octopus release, because the fix may involve tweaking the deployment parameters of the respective XAML deployment build definition. Hence we may be forced to restart the XAML deployment build, which never compiles code.
EDIT 2
First of all, why do we drive the deployment from TFS XAML builds rather than from Octopus? Historic reasons. We did not have Octopus at first. The deployment was done by our ad hoc code. When we introduced Octopus we decided to keep the XAML deploymenet builds for two reasons:
To save the cost of migrating all the XAML deployment builds with all the gazillion deployment parameters to Octopus. Maybe it was a wrong decision, maybe we could have automated the migration.
Because TFS has better facility to display test results. The deployment may end with deployment smoke tests and their results has to be published somewhere. We do not see how Octopus can help us publish the results, TFS can.
Why would one redeploy? For example, one of the deployment parameters is certificate thumbprint. When the certificate is renewed, this parameter must be changed (we do have automation for updating XAML build parameters). But often we discover that it was already deployed with wrong thumbprint. So, we fix the deployment and redeploy. Or, we discover some strange behavior of the deployed application and wish to redeploy with some extra tracing/debugging features.
There is a lot to unpack here, but I'll give it a go.
TL;DR It's the way you version the releases that's causing you all the pain. Change that and everything else will fall in to place
Lets start at the end and work backwards.
Octopus Deploy has a concept of Environments. This means that you can Deploy the same project to multiple environments and use Octopus's scoping mechanism to manage environment specific configuration.
So using your example.
Creates 3 packages from the build artifact previously unpacked by the
XAML build to match three areas of deployment - web farm, background
job engine cluster and the database
I set up an Environment in Octopus for each of your 50 Environments. (I'll use 3 environments in the example to keep it simple, but the principles apply no matter how many environments you have)
In my Dev Environment I have a single server so I create an environment called "Dev" and add the tentacle for that specific server. Then I tag the tentacle with the deployment type "Web", "Job", "Database"
I then set up a test environment which has 3 servers so I create the Environment and add the 3 servers. I then tag each tentacle with the deployment type "Web", "Job", "Database"
Finally I set up the Production environment. This has 5 web servers, 1 job server and 1 database server. I add all 7 tentacles to the environment, and tag them appropriately.
Now I only need 1 project to deploy to all 3 environments. In my project I have 3 steps.
Step 1 Deploy Web Site
Step 2 Deploy Jobs
Step 3 Deploy database
I can tag each of these steps to say what kind of tentacle I want to deploy to. Now when I run the deployment the link between the tags on the step, and the tags on the tentacle mean Octopus knows where to deploy the code.
Variables: Your variables can be scoped to an environment. So for example if your dev environment database connection string is dev.database.net/Instance and your test environment database connection string is test.Database.net/Instance then you can scope these in the variables section of the project. If your DNS is consitant with your environment names you could even use some of the built in variables to make adding environments more easy. i.e. ${Octopus.Environment.Name}.Database.net/Instance
Releases and version numbers: So here is where I think your problem lies. Adding the environment name to the release and trying to create multiple releases with the same version is basically causing you all of the pain.
Using the example above this gives us 55.0.18709.3-atwfm, but
sometimes we want to deploy the same build artifact in the same
deployment environment twice. But the only Octopus project would
already have the release 55.0.18709.3-atwfm, so how to deploy
55.0.18709.3 in atwfm again, without deleting the already existing release?
There are a couple of things here. In Octopus you can easily deploy again from the UI, however it sounds like you're rebuilding the artifact and trying to create a new release with the same version number. Don't do this! Each new build should have a distinct and unique build number / release number.
The principle I follow is "build once deploy many"
When you create a release it requires a version number, this release then flows through the environments. So I build my code and it gets a versions number 55.0.18709.3 then I deploy it to Dev. When the deployment has been verified I then want to "Promote" the release to test I can do this from within Octopus or I can get TFS to do this.
So I promote 55.0.18709.3 to test and then on to prod. If I need to know which release is in which environment, Octopus tells me this via the dashboard or API.
Finally I can "Orchestrate" the flow of releases through my environments using Build v.next.
So my end to end process looks something like.
Build vNext Build
Compile
Run Unit Tests
Package output
Publish package
build vNext Release
Call Octopus to create the release passing in the version number
Optionally deploy the release to the first environment on your way to live
I now have everything I need in Octopus to deploy to ANY environment with a single project and my environment specific configuration.
I can either "Deploy" the release to a specific environment or "Promote" the release from one environment to another. This can be done easily from within the Octopus UI
Or I can create a "Promote" using the Octopus plugin in TFS and use that to orchestrate the promotion of code through the environments.
Octopus Terminology.
Create release - This pulls together the Artifacts and Release number in Octopus to create an Immutable thing which will be deployed to one of more environments.
Deploy release - The act of pushing your code to a specific environment.
Promote release - Once the code has been deployed in to a single environment, it can them be promoted in to other environments
If you have a specific sequence of environments then you can use the "Lifecycles" feature of Octopus to enforce that workflow. but that's a topic for another day!
EDIT1 Response
I don't think the edit changes my answer, you can re-deploy the same release many times as you like. what you cannot do is create a new release with the same version number. You might want to decouple these steps could you add some more detail about what changes in the XAML build? You can change variables in a release, you can update them in octopus and then redeploy
EDIT 2 Response
That makes things clearer. I think you need to take the hit and migrate the parameters to Octopus. It's variable management is much better than XAML builds and although build vNext is comparable to Octopus it makes more sense to have the config in Octopus. As XAML builds are on their way out, it makes sense to move this stuff now. Whilst it might be a lot of work, at the end you'll have a much smother workflow and you can really take advantage of the power of Octopus.
The Test results point. I agree this is better suited to build vNext, so at this point you will be using build vNext as your Orchestrator and Octopus Deploy as your release management tool.
The process would look something like
Build vNext
Compile code.
Run Unit tests
Run Octopack
Publish packages
Call Octopus and Create release
Call Octopus to Deploy to "Dev"
Run Smoke tests
Run Integration Tests
Call "Selenium" to run Run UI tests
Call Octopus to Promote release to "Test"
Run Smoke tests
Run Integration Tests
Call "Selenium" to run Run UI tests
Call Octopus to Promote release to "Production" (Perhaps with a manual innervation)
Run Smoke tests
Run Integration Tests
Call "Selenium" to run Run UI tests
I'm trying to setup a release to three environments, Dev, QA, Production, and working through failures, mainly IIS Application errors.
Right now, I'm trying to get my QA environment setup. Unfortunately it is a lot of trial and error to get the release to pass. When it fails, I have to go edit the release, edit my tasks for the QA environment, then start a release all over.
My problem is, I'm just trying to get this setup, these aren't "real" releases. However I'm wasting time after each configuration change having it deploy code to my "Dev" environment. I already know those steps work.
Is there a way I can skip an environment, and select a specific environment I want it to release code too?
As Daniel said that you can choose the environments which you want to deploy manually when start a release.
You also can set Manual only for the environment directly:
You also can enable Artifact filters with specific tags or branch (git), then just the artifact meet the filters, the release can deploy to this environment.
When you start a release you're presented with a list of all of the environments and their deployment conditions. Simple set all of the environments to "Manual". Then when you create the release, you can choose the environment to which you wish to deploy from the "Deploy" menu.
Quick question.
Is there a way to constrain/restrict what order users can can deploy builds to environments?
For example if I have these four environments configured with manual push-button deploy (not-automated) I can start all four together if I want. I don't have to wait for the other to be done before kicking off the next one:
DEV
TEST
STAGE
PROD
Microsoft seems to be missing this feature in TFS 2015. It would make sense to offer a deployment condition that states that previous environments must have successful deployments before you can run push-button deploy for the next.
Yes, I know, you are going to say "but you can automate that so the deploys run in the order you want." Management here does NOT want that. They want push button deployment for each environment WITH a constraint that previous environments must be completed first.
This means a manual start for each environment.
Other than having the release manager "eyeball" the situation before pushing the button for the next environment I can't see a way to configure this rule.
Any ideas?
There is not any restriction on manually deploy situation for now. This is designed for giving you the ability to override the release process.
Note that you can always deploy a release directly to any of the
environments in your release definition by selecting the Deploy
action when you create a new release.
In this case, the environment triggers you configure, such as a
trigger on successful deployment to another environment, do not apply.
The deployment occurs irrespective of these settings. This gives you the ability to override the release process. Performing such
direct deployments requires the Manage deployments permission, which
should only be given to selected and approved users.
Source Link: Environment triggers
Suggest you use automation triggers, you could use Parallel forked and joined deployments, in combination with the ability to define pre- and post-deployment approvals, this enables the configuration of complex and fully managed deployment pipelines to suit almost any release scenario.
If you insist on manual push-button deploy, you may have to ask the release manager "eyeball" the situation to restrict environment deployment order as you mentioned.
I'm finalising my CI setup with TFS and I have one last part to overcome. Currently, every CI build is pushed to octopus with the nuget packages versioned as major.minor.patch.buildid, and these are used in releases in Octopus versioned as major.minor.patch-beta{buildid}. Therefore, every CI build will be tagged as a beta build for a given release, e.g. 1.3.0.194 / 1.3.0-beta194. There will no doubt be a number of iterative dev builds before one is chosen to go to the test team - let's assume the chosen build is 194 in this case. At this point I envisage the creation of a new build, 1.3.0-rc1, which uses the build 194 binaries. This is then pushed to the test environment and testing begins. There may be a number of testing cycles, so let's say the testers sign off the version 1.3.0-rc4. A new release could then be made, 1.3.0, based on the 1.3.0-rc4 binaries, which is the gold release for the product.
Firstly, is this a good idea? Some feedback would be much appreciated.
Seperately, is it possible to restrict deployments to certain environments based on a tag in the version? In my example, I would never want a release marked as -beta to be deployed to a test environment - only -rc builds should be. Likewise, only tag-less builds should be deployed to production environments.
Here is the reason why you definitely shouldn't do that.
When you create version 1.3.0.194, Octopus will guarantee that the artefacts, process, and variables are given a snap-shot that means the deployment will performed in the same way on every environment.
You can edit these things as much as you like, but 1.3.0.194 will not become a more risky deployment due to these changes, as it will be oblivious to them.
If you created version 1.3.0.194, then made changes to your deployment process, or variables, these changes would leak into the version 1.3.0.194-beta and it would no longer be the same deployment that you have tested. The same would occur when you change to version 1.3.0.194-rc - more changes would leak in that you haven't actually tested.
Your deployment process, just like your code, should be versioned and tested - and that is what you get by using the same version throughout your deployment lifecycle.
Tagging deployment packages with beta/rc is a bad idea because this introduces an extra step in your delivery process. You will have many built versions some will advance to upper environments, some won't. That's okay. Your trying to treat these like regular Nuget packages used for dependency versioning/management. Its different. Just increment your build numbers without the tags and promote the builds that get sign off.
You shouldn't restrict deployments to environment based on tags either.
I'm trying to set up a deployment in Octopus Deploy. As many other devs, I do that through iterations: tweak steps/scripts/packages -> click "Deploy" -> inspect the outcome -> start again if not satisfied. In case of Octopus Deploy, it uses a snapshot of the deployment process if any steps/scripts were changed since last release. Basically, when it comes to deploy, I get this warning:
Warning: for consistency, this deployment will use a snapshot of the variables and deployment process, which do not include the latest changes that have been made to the project. A changed process can only be incorporated by creating a new release (this one may be renamed if desired). Variables can be updated via the release page.
This means that I need to add a new release just to check if my deployment scripts change has taken effect or not. To add a release, I need to update the version and so on.
My question would be: how can I redeploy the same release with all the step changes introduced since the previous release? Is there a way to not create a new release to do that?
I asked this question of Paul Stovall several months ago when I started working with Octopus. His answer was "no".
I'm afraid there is not a way to do this without creating a new release. When you create a release, Octopus deploy stores an object in its internal database which is a snapshot in time containing everything needed to deploy. It doesn't have a mechanism to update these stored objects, and so it is required for you to create a new release each time.
Please note that even though a snapshot of the process is taken, the variables can be updated if you're redeploying a previous release. If you're just changing variable values, then yes, you can reuse the same release over and over. If you're changing any of the process steps, then no - you have to create a new release.
If anything the version number just helps keep things segregated. The only time this becomes a problem is if you're using NuGet versioning, but if you're using the standard variable template versioning in Octopus deploy, it's incrementing the last digit in the version number.
You can rename your failing release on the release page (under: Edit this release). That way you can create a new release with the old versionnumber and avoid unecessary work.