I would like to use a bamboo plan variable in a deployment project.
I inject the version of the project into ${bamboo.inject.version} during the build plan, and I would like to use it in the deployment project. (The artifact I am deploying uses the version in its name).
I've tried referencing ${bamboo.planRepository.inject.version} and ${bamboo.inject.version} but neither of these work.
I am happy to hear suggestions of other ways of doing this too.
Edit: I did manage to achieve the results I wanted by adding the properties file as a build artifact, then exporting it into the deployment project. This seems rather roundabout but works. Any other ideas appreciated!
As suggested by ToAsT, set the scope of your injected variables to Result first, then add a variable inject.version into your plan, and make sure you also have ${bamboo.inject.version} for release versioning field of the deployment project.
You could potentially add that version info into the release name itself, but if what you're doing works then stick with that. Check out what's offered in the free plugins.
Yes you can, for that you need to use a task called Inject Variables.
In your Build Plan Task creates a key=value file (can be a script task):
echo "DEPLOYMENT_VERSION={your_version_here}" > /etc/.version
In the next task, use the Inject Variables task to export this variable through your build plan and deployment projects (that is related):
# this is the yaml used in bamboo-specs
version: 2
# ...
tasks:
- inject-variables:
file: /etc/.version
scope: RESULT # share between stages and deployment releases
namespace: anything_you_want
When you run your deployment task, you can call the variable with the following name:
echo ${bamboo.anything_you_want.DEPLOYMENT_VERSION}
Sources:
https://confluence.atlassian.com/bamboo/configuring-a-variables-task-687213475.html
https://docs.atlassian.com/bamboo-specs-docs/7.0.1/specs.html?yaml#task-inject-variables
Related
We have a release definition which delivers a bunch of asp.net core services along with an Angular app.
Most service are not updated very often so the question is how to compare an artifact version with already deployed into an environment and skip if the latest version had been deployed before?
We have multiple environments in the pipeline.
I dont think it is possible, at least natively, you can calculate file hashes and dont deploy if they match, another option would be using path triggers to filter when an app is build. for example, your directory structure looks like this:
root
|--app1
|--app2
etc
you can define path filters in your yaml build like this:
trigger:
paths:
include:
- app1/*
- sharedlibs/* (if you have them)
this way build will only trigger if there are any changes to files in those directories
You can add additional release environment to check current artifact version through PowerShell (e.g. Build.SourceVersion, check variables in release), then fail task if there was already successfully released.
For Staging environment, choose After environment option and select previous environment.
On the other hand, since you have mentioned most service are not updated very often, you could use 4c74356b41's suggestion to filter build, to only build and release the changes you want.
I have different config files for each different deployment environments I have. Dev, QA and live are those environments.
How do i get bamboo to use the right config file for each environment when deploying?
So when deploying the dev environment bamboo use the dev config file and change name on it and put it on the right place.
I assume that artifacts can fix this? But how?
I solved it. I simply used artifacts and pointed directly to the file. Then in deploy i made a ps script for deploying the right artifact to the right deployment environment.
I also made the file changing depending on environment so i don't need separately config files.
Just hit me up if you are having the same problem, Atlassian documentation on this is terrible.
My question is about maintainability of vNext/Octopus processes vs XAML based processes. Or rather about the impossibility to maintain them sanely leading me to believe we are doing something terribly wrong.
Given:
Microsoft pushes to phase out its TFS XAML builds in favour of the vNext builds
Octopus Deploy is a popular deployment automation framework
We have many XAML based builds, but starting to port to vNext
The deployments are automated with Octopus Deploy
Concretely, we have three kinds of builds going on in QA:
Old XAML based compilation builds producing artifacts to be deployed
Ultimately just builds the code, zips it and places in a well-known location
New vNext compilation builds producing artifacts to be deployed
Same as above
Deployment builds
XAML based build definition per deployment environment. This is the source of truth for the particular deployment, containing all the configuration URLs, connection strings, certificate thumbprints, etc.. The build definition has over 100 build parameters. Each time a new environment is setup we clone an existing XAML build definition and change the parameters.
This build unpacks the build artifact, generates all the web/app config files based on the configuration parameters and kicks off Octopus Deploy with a lot of parameters using octo.exe and waiting for the end
Octopus Deploy process
Creates 3 packages from the build artifact previously unpacked by the XAML build to match three areas of deployment - web farm, background job engine cluster and the database
Delivers the relevant packages to the relevant tentacles.
The tentacles unpack and setup their respective packages
So, if we have 50 deployment environments, then we have 50 XAML deployment builds, each capturing the context of the respective environment. But the XAML deployment build delegates the deployment job to Octopus and here we are forced to have 50 Octopus projects - one per deployment.
Why is it so? We examined the option of having just one Octopus Project, but what would be the Release versions of such project? In order for us to be able to navigate amongst the gazillion releases, the release version must include:
The build version of the deployed code, e.g. 55.0.18709.3
The name of the deployment environment, e.g. atwfm
Using the example above this gives us 55.0.18709.3-atwfm, but sometimes we want to deploy the same build artifact in the same deployment environment twice. But the only Octopus project would already have the release 55.0.18709.3-atwfm, so how to deploy 55.0.18709.3 in atwfm again, without deleting the already existing release?
We could not find a workaround and so, we have Octopus project per deployment.
THIS IS ABSOLUTELY CRAZY because Octopus projects are a pain to update. Suppose we need to add a step - go do it in 50 projects. There are great advises on the Internet to use automation to edit multiple projects. Not ideal at all.
vNext, BTW, has the same problem. If I am to port the existing XAML builds to vNext I will end up with 50 vNext deployment builds. If I decide to add a step, I need to do it in all the 50 builds!!!
Note, that XAML builds do not have this problem (they have many others, though), because their the process is separate from the parameters. I can modify the workflow once and all the XAML builds are now updated with the new process change.
My question is - how do people work with vNext and Octopus, because our process drives me crazy. There must be a better way.
EDIT 1
I would like to clarify. We sometimes want to deploy the same build artifacts twice. We are not recompiling them and reusing the same version. No. We already have the build artifacts handy with the build version baked inside the artifact. We may want to deploy it the second time into the same environment because, for example, some databases in that environment have been misconfigured and now this is fixed and we need to redeploy. This does not mean we can rerun the already existing Octopus release, because the fix may involve tweaking the deployment parameters of the respective XAML deployment build definition. Hence we may be forced to restart the XAML deployment build, which never compiles code.
EDIT 2
First of all, why do we drive the deployment from TFS XAML builds rather than from Octopus? Historic reasons. We did not have Octopus at first. The deployment was done by our ad hoc code. When we introduced Octopus we decided to keep the XAML deploymenet builds for two reasons:
To save the cost of migrating all the XAML deployment builds with all the gazillion deployment parameters to Octopus. Maybe it was a wrong decision, maybe we could have automated the migration.
Because TFS has better facility to display test results. The deployment may end with deployment smoke tests and their results has to be published somewhere. We do not see how Octopus can help us publish the results, TFS can.
Why would one redeploy? For example, one of the deployment parameters is certificate thumbprint. When the certificate is renewed, this parameter must be changed (we do have automation for updating XAML build parameters). But often we discover that it was already deployed with wrong thumbprint. So, we fix the deployment and redeploy. Or, we discover some strange behavior of the deployed application and wish to redeploy with some extra tracing/debugging features.
There is a lot to unpack here, but I'll give it a go.
TL;DR It's the way you version the releases that's causing you all the pain. Change that and everything else will fall in to place
Lets start at the end and work backwards.
Octopus Deploy has a concept of Environments. This means that you can Deploy the same project to multiple environments and use Octopus's scoping mechanism to manage environment specific configuration.
So using your example.
Creates 3 packages from the build artifact previously unpacked by the
XAML build to match three areas of deployment - web farm, background
job engine cluster and the database
I set up an Environment in Octopus for each of your 50 Environments. (I'll use 3 environments in the example to keep it simple, but the principles apply no matter how many environments you have)
In my Dev Environment I have a single server so I create an environment called "Dev" and add the tentacle for that specific server. Then I tag the tentacle with the deployment type "Web", "Job", "Database"
I then set up a test environment which has 3 servers so I create the Environment and add the 3 servers. I then tag each tentacle with the deployment type "Web", "Job", "Database"
Finally I set up the Production environment. This has 5 web servers, 1 job server and 1 database server. I add all 7 tentacles to the environment, and tag them appropriately.
Now I only need 1 project to deploy to all 3 environments. In my project I have 3 steps.
Step 1 Deploy Web Site
Step 2 Deploy Jobs
Step 3 Deploy database
I can tag each of these steps to say what kind of tentacle I want to deploy to. Now when I run the deployment the link between the tags on the step, and the tags on the tentacle mean Octopus knows where to deploy the code.
Variables: Your variables can be scoped to an environment. So for example if your dev environment database connection string is dev.database.net/Instance and your test environment database connection string is test.Database.net/Instance then you can scope these in the variables section of the project. If your DNS is consitant with your environment names you could even use some of the built in variables to make adding environments more easy. i.e. ${Octopus.Environment.Name}.Database.net/Instance
Releases and version numbers: So here is where I think your problem lies. Adding the environment name to the release and trying to create multiple releases with the same version is basically causing you all of the pain.
Using the example above this gives us 55.0.18709.3-atwfm, but
sometimes we want to deploy the same build artifact in the same
deployment environment twice. But the only Octopus project would
already have the release 55.0.18709.3-atwfm, so how to deploy
55.0.18709.3 in atwfm again, without deleting the already existing release?
There are a couple of things here. In Octopus you can easily deploy again from the UI, however it sounds like you're rebuilding the artifact and trying to create a new release with the same version number. Don't do this! Each new build should have a distinct and unique build number / release number.
The principle I follow is "build once deploy many"
When you create a release it requires a version number, this release then flows through the environments. So I build my code and it gets a versions number 55.0.18709.3 then I deploy it to Dev. When the deployment has been verified I then want to "Promote" the release to test I can do this from within Octopus or I can get TFS to do this.
So I promote 55.0.18709.3 to test and then on to prod. If I need to know which release is in which environment, Octopus tells me this via the dashboard or API.
Finally I can "Orchestrate" the flow of releases through my environments using Build v.next.
So my end to end process looks something like.
Build vNext Build
Compile
Run Unit Tests
Package output
Publish package
build vNext Release
Call Octopus to create the release passing in the version number
Optionally deploy the release to the first environment on your way to live
I now have everything I need in Octopus to deploy to ANY environment with a single project and my environment specific configuration.
I can either "Deploy" the release to a specific environment or "Promote" the release from one environment to another. This can be done easily from within the Octopus UI
Or I can create a "Promote" using the Octopus plugin in TFS and use that to orchestrate the promotion of code through the environments.
Octopus Terminology.
Create release - This pulls together the Artifacts and Release number in Octopus to create an Immutable thing which will be deployed to one of more environments.
Deploy release - The act of pushing your code to a specific environment.
Promote release - Once the code has been deployed in to a single environment, it can them be promoted in to other environments
If you have a specific sequence of environments then you can use the "Lifecycles" feature of Octopus to enforce that workflow. but that's a topic for another day!
EDIT1 Response
I don't think the edit changes my answer, you can re-deploy the same release many times as you like. what you cannot do is create a new release with the same version number. You might want to decouple these steps could you add some more detail about what changes in the XAML build? You can change variables in a release, you can update them in octopus and then redeploy
EDIT 2 Response
That makes things clearer. I think you need to take the hit and migrate the parameters to Octopus. It's variable management is much better than XAML builds and although build vNext is comparable to Octopus it makes more sense to have the config in Octopus. As XAML builds are on their way out, it makes sense to move this stuff now. Whilst it might be a lot of work, at the end you'll have a much smother workflow and you can really take advantage of the power of Octopus.
The Test results point. I agree this is better suited to build vNext, so at this point you will be using build vNext as your Orchestrator and Octopus Deploy as your release management tool.
The process would look something like
Build vNext
Compile code.
Run Unit tests
Run Octopack
Publish packages
Call Octopus and Create release
Call Octopus to Deploy to "Dev"
Run Smoke tests
Run Integration Tests
Call "Selenium" to run Run UI tests
Call Octopus to Promote release to "Test"
Run Smoke tests
Run Integration Tests
Call "Selenium" to run Run UI tests
Call Octopus to Promote release to "Production" (Perhaps with a manual innervation)
Run Smoke tests
Run Integration Tests
Call "Selenium" to run Run UI tests
I am setting up Continuous Deployment for a web app and I'm having trouble doing config transformations. The build and release both work without config transforms, but I would like to add a config transform step to ensure the correct connection strings, storage accounts, etc are in the web config.
I am using the config transform task in VSTS. The artifact that is getting built during the build process is a zip file. I am then extracting those files, and it gives an extremely deep file structure. The config transform works if I hardcode the file structure, but I worry that this could change, so I'd like to generalize it. If I do $(mypath)/**/Web.config it does not error and the logs show that it is doing transforms, but once it finishes, I look at the file in Azure and can see that the transforms weren't actually done or saved.
My requirements for this are that I can't use xslt transforms or the XML transforms(web.environment.config) because we would like to keep the important values out of source control and in VSTS. We are also doing config transforms on NLog.config, and those transforms are setup the same way and seeing the same issues. Any help on getting this more general path to work, or a different way of solving this would be greatly appreciated.
Other potentially helpful information:
MSBuild Arguments in Build Solution step:
/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"
Path Structure
$(My.Paths.InstallDirectory)\Content\d_C\a\1\s\Apps\ProjectName\ProjectName\obj\Release\Package\PackageTmp\Web.config
There is File Transforms & Variable Substituion Options in Azure App Service Deploy task, so you can use XML variable substitution or JSON variable substitution to do it without extracting the package file (zip file):
For example:
Add a new variable in Release definition (Scope: select an environment), for example: Name: DefaultConnection; Value: Data source=xxx…), clicking lock icon to change variable type to secret.
Check XML variable substitution option in Azure App Service Deploy task
Then during the deploying, any config and parameters.xml files will be updated with that variable value if the key or name entries in appSettings, applicationSettings and connectionStrings sections are matched the variable name
Related article: XML variable substitution
Another way is that you can change the value in web deploy parameters file (SetParameters.xml) through Token replace task (You also can specify the different file in Azure App Service Deploy task per to Environment of release, such as SetParameters.Dev.xml, SetParameters.QA.xml)
An article about Configuring Parameters for web package deployment
BTW: You can remove the source structure from web deploy package by specifying /P:PackageTempRootDir="" msbuild argument
I have a Service Fabric project set up with CD in VSTS. The CD process have been set up from the CD wizard in visual studio. So far so good but when the release is made I'm not getting any value from the pre-defined variable called Build.DefinitionName that I use for naming the releases.
This is how my Release name is configured: $(Build.DefinitionName)-$(date:yyyyMM).$(rev:r)
If we look in the release log there is no value for the variables:
[BUILD_DEFINITIONNAME] --> []
[RELEASE_ARTIFACTS_{Primary artifact alias}*_DEFINITIONNAME] --> []
*This value is populated correctly.
And according to the docs of primary artifact variables the two variables above should be the same.
As a result of this my releases are named $(Build.DefinitionName)-201702.7
If I use Build.BuildNumber instead. I get the correct value.
How can I get the variables populated?
Update
When creating the Build and Release definitions manually the $(Build.DefinitionName) gets populated correctly on the Release side. But the problem seems to appear when you use the CD wizard from Visual Studio. I might be missing something but the settings of the Build Definition are identical. Or there is some funky stuff going on with the CD wizard.
The solution is to create the release definition manually on web access and set Continuous Deployment.
On the build number is passed from Build to a release, there is no out of the box way to pass more variables between build and release.
However I write a ser of build tasks to do this: https://marketplace.visualstudio.com/items?itemName=nkdagility.variablehydration
It consists of two tasks, the first saves the specified variables to a json file, which you should put in the build output.
The second restores the variables you want. Especially useful if you have multiple source build for your release.